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FORE WORD 

A Computation Seminar, sponsored by the Inter- 
w national Business Machines Corporation, was 
held in the IBM Department of Education, Endicott, 
New York, from December 5 to 9, 1949. Attending the 
Seminar were one hundred and seven research engineers 
and scientists who are experienced both in applying 
mathematical methods to the solution of physical prob- 
lems and in the associated punched card methods of 
computation. Consequently, these Proceedings represent 
a valuable contribution to the computing art. The Inter- 
national Business Machines Corporation wishes to ex- 
press its appreciation to all those who participated in 
this Seminar. 
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A MAJOR CONCERN which is frequently voiced in 
connection with very fast computing machines, particu- 
larly in view of the extremely high speeds which may now 
be hoped for, is that they will do themselves out of busi- 
ness rapidly ; that is, that they will out-run the planning 
and coding which they require and, therefore, run out of 
work. 

I do not believe that this objection will prove to be valid 
in actual fact. It is quite true that for problems of those 
sizes which in the past— and even in the nearest past — 
have been the normal ones for computing machines, plan- 
ning and coding required much more time than the actual 
solution of the problem would require on one of the hoped- 
for, extremely fast future machines. It must be considered, 
however, that in these cases the problem-size was dictated 
by the speed of the computing machines then available. 
In other words, the size essentially adjusted itself auto- 
matically so that the problem-solution time became longer, 
but not prohibitively longer, than the planning and coding 
time. 

For faster machines, the same automatic mechanism 
will exert pressure toward problems of larger size, and the 
equilibrium between planning and coding time on one 
hand, and problem-solution time on the other, will again 
restore itself on a reasonable level once it will have been 
really understood how to use these faster machines. This 
will, of course, take some time. There will be a year or 
two, perhaps, during which extremely fast machines will 
have to be used relatively inefficiently while we are finding 
the right type and size problems for them. I do not be- 
lieve, however, that this period will be a very long one, and 
it is likely to be a very interesting and fruitful one. In 
addition, the problem types which lead to these larger sizes 
can already now be discerned, even before the extreme 
machine types to which I refer are available. 

Another point deserving mention is this. There will 
probably arise, together with the large-size problems which 

*This is a digest of an address presented at the IBM Seminar on 
Scientific Computation, November, 1949. 



are in "equilibrium" with the speed of the machine, other 
and smaller, "subliminal" problems, which one may want 
to do on a fast machine, although the planning and pro- 
gramming time is longer than the solution time, simply 
because it is not worthwhile to build a slower machine for 
smaller problems, after the faster machine for larger 
problems is already available. It is, however, not these 
"subliminal" problems, but those of the "right" size which 
justify the existence and the characteristics of the fast 
machines. 

Some problem classes which are likely to be of the 
"right" size for fast machines are of the following : 

1. In hydrodynamics, problems involving two and three 
dimensions. In the important field of turbulence, in par- 
ticular, three-dimensional problems will have to be pri- 
marily considered. 

2. Problems involving the more difficult parts of com- 
pressible hydrodynamics, especially shock wave formation 
and interaction. 

3. Problems involving the interaction of hydrodynamics 
with various forms of chemical or nuclear reaction 
kinetics. 

4. Quantum mechanical wave function determinations 
— when two or more particles are involved and the prob- 
lem is, therefore, one of a high dimensionality. 

In connection with the two last-mentioned categories of 
problems, as well as with various other ones, certain new 
statistical methods, collectively described as "Monte Carlo 
procedures," have recently come to the fore. These require 
the calculation of large numbers of individual case his- 
tories, effected with the use of artificially produced "ran- 
dom numbers." The number of such case histories is neces- 
sarily large, because it is then desired to obtain the really 
relevant physical results by analyzing significantly large 
samples of those histories. This, again, is a complex of 
problems that is very hard to treat without fast, automatic 
means of computation, which justifies the use of machines 
of extremely high speed. 
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Some Nlethods of Solving Hyperbolic and Parabolic 
Partial Differential Equations 



RICHARD W. HAMMING 

Bell Tele f hone Laboratories 



THE MAIN PURPOSE of this paper is to present 
a broad, non-mathematical introduction to the general field 
of computing the solutions of partial differential equations 
of the hyperbolic and parabolic types, as well as some re- 
lated classes of equations. I hope to show that there exist 
methods for reducing such problems to a form suitable for 
formal computation, with a reasonable expectation of ar- 
riving at a usable answer. 

I have selected four particular problems to discuss. 
These have been chosen and arranged to bring out certain 
points which I feel are important. The first problem is 
almost trivial as there exist well-known analytical methods 
for solving it, while the last is a rather complicated partial 
differential-integral equation for which there is practically 
no known mathematical theory. 

To avoid details, I shall give only a brief introduction to 
the physical situation from which the equations came. Nor 
shall I dwell at all on the importance or meaning of the 
solutions obtained. 

Lastly, I have chosen only equations having two inde- 
pendent variables, usually a space variable and a time 
variable. Similar methods apply to equations having three 
and more independent variables. 

I have not attempted to define rigorously what is meant 
by hyperbolic or parabolic partial differential equations, 
nor shall I later. Instead, I intend to bring out certain 
common properties, and inferentially these properties de- 
fine the classes of equations. In fact, from a computer's 
point of view it is the class of problems which is amenable 
to the same type of attack that provides the natural classi- 
fication. It is on this basis that I have included a partial 
differenial-integral equation as the last example. 

Each of the four problems is carried successively further 
toward its solution until, in the last example, I have given 
the detailed steps which were actually used. 

If, in the rest of the paper, I do not mention any names, 
it should not be inferred that I did everything alone; on 
the contrary, I have at times played but a minor part in 
the entire effort. 



The Wave; Equation 

The classic and best known example of a hyperbolic 
partial differential equation in two independent variables 
is the wave equation : 






1 d^w 



This is the equation which describes the propagation of 
signals, w, in one dimension, x. The signals progress in 
time, t, with a velocity, c. This equation is a linear equation 
and, as such, there is a large body of theory available for 
use in solving it. Thus, it is not likely that anyone would 
be called upon to solve it numerically except in very un- 
usual circumstances. Nevertheless, I have chosen it as my 
first example, since I hope its simplicity and familiarity to 
you will aid in bringing out the main points I wish to make. 

In solving partial differential equations it is customary 
to replace the given equations with corresponding differ- 
ence equations, and then to solve these difference equa- 
tions. Whether one looks at the approximation as being 
made once and for all and then solving the difference 
equations as exactly as possible, or whether one looks at 
the difference equations as being an approximation at every 
stage is a matter of viewpoint only. I personally tend to 
the latter view. 

In the case at hand, the second differences are clearly 
used as approximations to the second derivatives. Such a 
choice immediately dictates an equally spaced rectangular 
net of points at which the problem is to be solved. Such a 
net is shown in Figure 1. The space coordinate, x, is ver- 
tical while the time coordinate, t, is horizontal. Thus, at a 
fixed time, t, we look along the corresponding vertical line 
to see what the solution is in space, x. 

Suppose for the moment that a disturbance occurs at 
the upper point at time t. As time goes on the disturbance 
will spread out in space as shown in the figure. The space 
covered by the disturbance at any later time is indicated 
by the length of the corresponding vertical shading line at 
that time. The area of this disturbance in the figure is 
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Figure; 1. Wavf; Equation 
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called the "cone of disturbance." The slopes of the bound- 
ing lines indicate the velocity of propagation c, and in this 
simple case they are straight lines. In the mathematical 
theory of partial differential equations the lines are called 
"characteristics." 

The figure shows a second disturbance started at the 
same time, t, but at a lower point. This, too, spreads out as 
time goes on, and there finally occurs a time when the two 
cones overlap. 

Consider, again, the given equation. The second differ- 
ence in the x direction is calculated from the three points 
which are connected by the vertical line. This is to be 
equated to \/c^ times the second difference in the time 
direction, which naturally uses the three solid black points. 
Suppose that the solution of the problem, up to the time t, 
is known, then we have an equation giving an estimate of 
the solution at one point at a later time, t -\- i\t. 

Suppose, now, that the spacing in the x direction is kept 
as before, but the spacing in t is increased so as to predict as 
far as possible into the future. It should be obvious that the 
spacing in t cannot increase so far that the advanced point 
falls into the cones of disturbance of the first two points 
which are neglected. To do so is to neglect effects that 
could clearly alter the estimate of what is going to happen 
at such a point. Thus, it is found that, for a given spafcing 
in the x direction, and a formula of a given span for esti- 
mating derivatives in the x direction, there is a maximum 



permissible spacing in the t direction, beyond which it is 
impossible to go and still expect any reasonable answer. 
In this simple case the condition may be written 

C 

Supposing that this condition has been satisfied, and 
also, the solution up to some time t is known, the above 
method may be used to advance the solution a distance A^ 
at all interior points. A point on the boundary cannot be so 
advanced. There must be independent information as to 
what is happening on the boundary. Such conditions are 
called "boundary conditions" and are usually given with 
the problem. The simplest kind of boundary condition 
gives the values of the dependent variable w by means of 
a definite formula. More complex situations may give only 
a combination of a function w and its derivative dw/dx. 
Such situations may require special handling when solving 
the problem in a computing machine, but in principle are 
straightforward. 

A step forward at all points x is usually called a "cycle," 
and the solution of a problem consists of many, many 
repetitions of the basic cycle. The sole remaining question 
is that of starting the solution for the first cycle or two. 
Just as in the case of ordinary differential equations, this 
usually requires special handling and is based on some 
simple Taylor's series expansion of the solution. In prac- 
tice, this step is often done by hand before the problem is 
put onto whatever machines are available. 

As remarked before, this problem is not realistic, so 
having made a few points about spacing, boundary condi- 
tions, and initial, or starting, conditions, let us turn to a 
more complex problem. 

The; Two-Be;am Tube; 

The two-beam tube is a tube with two electron beams 
going down the length of it together. Upon one beam of 
electrons a signal is imposed. The second beam, which has 
a slightly greater velocity, interacts with the first beam 
through their electric fields, and may be regarded as giving 
up some of its energy to the first beam. This in turn pro- 
duces, one hopes, an amplification of the signal on the 
first beam. 

The equations describing one particular idealization of 
such a tube are : 
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where the solution is to be periodic in time of period 1, 
and we are given information as to the state of affairs at 
the beginning of the tube, x = 0. The upper two equations 
for i = 1 describe one of the beams, while for i = 2 they 
describe the other beam. The lower two equations describe 
the interaction between the two beams of electrons. 

I shall gloss over any questions of existence theorems 
for such a system and merely suppose that there is a solu- 
tion. The information needed to start the problem at ;r = 
comes from the " linear" theory which is not hard to find 
from a "linearized" form of the equations. We are here 
called upon to calculate the essentially nonlinear aspects 
of the tube. 

The first reduction of the equations has already been 
performed before they were written as above, namely, that 
of transforming out of the equations all of the various 
constants and parameters of the problem that we could. 
In their present state the Vi of the equations give the ve- 
locities of the two beams measured in units of the mean 
velocity, the pi the corresponding charge densities of the 
beams measured in mean charge density units, while the 
^ and ^ describe the electric field in suitable dimension- 
less units. 

Since we are expecting a "wave-like" solution, it is 
convenient to transform to a moving coordinate system 
which moves with the expected mean velocity of the two 
beams. In such a coordinate system, the dependent vari- 
ables Pi, Vi, $ and * may be expected to change slowly. 

The equations obtained by such a transformation, 
X = a- <T = X 

or 

t = T -\- <J T — t — X , 

are 

T-[pi^i] =T-[pi{'^i- 1)] 

|;M=^[(e'.-l)']+* 



da 



OT 



where the solution is still periodic in time t with period 1. 
In solving the usual hyperbolic type of equation, one 
advances step by step in time, but in this problem a pe- 
riodic condition in time is given on the solution, and were 
the time to be advanced, it would be difficult to make the 
solution come out periodic. There would also be difficulty 
in finding suitable starting conditions. Instead of advancing 
in time, advancement is step by step down the length of 
the tube in the a direction, using the periodic condition in 
T to help estimate the derivatives in the t direction at the 
ends of the interval. Thus, the periodic condition in effect 
supplies the boundary conditions. 



One may calculate, if one wishes, the characteristic lines 
and determine the cones of disturbance, but in this case it 
must be looked at sidewise, as it were. Assuming that the 
solution is known for an interval of time, how far in space 
may the solution be predicted at a time corresponding to 
the mid-point of the time interval ? If the cones of disturb- 
ance were to be calci^lated, it would be found, as is usual in 
nonlinear problems, that the velocity of propagation de- 
pends on the solution which is to be calculated. For exam- 
ple, a large shock wave of an explosion travels at a velocity 
that depends not only on the medium through which it 
passes, but also upon the amplitude of the shock wave itself. 

Let us turn to the question of choosing a net of points 
at which we shall try to calculate an approximate solution. 
The use of a two-point formula in the t direction for esti- 
mating the T derivatives requires a great many points and 
produces a very fine spacing in the o- direction. If a four- 
point formula is chosen (a three-point one is hardly better 
than a two-point one for estimating first derivatives), the 
following is obtained. 
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A formula like this is easiest to obtain by expanding each 
term about the mid-point of the interval in a Taylor's 
series with a remainder. Since in this moving coordinate 
system we expect a sinusoidal variation in time, the fifth 
derivative is estimated from the function 

/ = sin 27rT . 
In order to obtain an accuracy of about 1 part in 10* as 
the maximum error, it is necessary to choose 24 points in 
the T direction — a most fortunate choice in view of the 
product 24At in the denominator of the estimate. 

The statement that the maximum error at any one point 
is at most one part in 10* tells very little about the accu- 
mulated error due to many, many small errors, but as far 
as I know, there are no general methods which are both 
practical and close enough to help out in this type of situ- 
ation. My solution to this dilemma is twofold: 

1. To the person who proposed the problem, I pose 
questions such as, "If the solution is disturbed at a 
given point, will it tend to grow, remain the same, 
or die out?" At the same time, I try to answer the 
question independently, keeping an eye on the rou- 
tine to be used in the solution of the problem. 

2. Hope that a properly posed problem does have a 
solution, and that human ingenuity is adequate to 
the problem at hand. 

Such a method may lead one astray on occasions, but with 
nothing else to fall back on, I feel that it is better than 
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inactivity. One should, of course, make a diligent effort to 
resolve this dilemma, but pending that time, go ahead and 
hope for the best, ready at any time to suspect the solu- 
tions obtained. 

Returning to our problem, some of the advantages of a 
four-point formula for estimating the derivatives in the t 
direction have been listed. Let us look at the disadvantages 
in general. In the first place, except in periodic cases such 
as this one, where one can double back the solution from 
the top of the interval to add on to the bottom, the difficult 
problem of estimating derivatives near the boundaries 
arises. In the second place, one faces the task of assem- 
bling information from four different points to estimate 
the derivative at any one point. If, for example, the four 
values lie on four different IBM cards, it is not easy to 
get the information together. One method would be to 
calculate both 1 and 27 times the value of the function on 
each card and then on one pass through the accounting 
machine-summary punch equipment, using selectors and 
four counters to accumulate the running sums, punch out 
the proper totals along with suitable identification on each 
summary punched card. 



To estimate the derivatives on the a direction to advance 
one step, a simple two-point formula is used. Since both 
the four- and two-point formulas give estimates at the 
mid-points of the intervals, one is led to a "shifting net" 
of points as shown in Figure 2. Such a net leads to some 
slight troubles in the identification of points, but gives 
probably the least calculation where it is necessary to deal 
with many odd order derivatives. At least in this case, it 
certainly does. I have glossed over the accuracy of the 
estimate of the derivative in the o- direction, but in this 
case it was adequate, due to the fineness in the spacing 
necessary to satisfy the net spacing condition in Ao- and 
At. 

Let us drop this problem at this point and take up the 
next example. 

A Parabowc Partial Diffe;re;ntial Equation 

The most common parabolic partial differential equation 
in two independent variables has the form 

dB Attj d^H 

dt ~ c^ dx^ ' 

Such an equation describes the flow of heat, the diffusion 
of material, and the magnetization of iron. 

In the particular case we shall discuss, a thin slab of 
permalloy is given, 2 mils thick and of infinite extent in 
the other two directions. This slab is subjected to an ex- 
ternal field H which is changing in a sinusoidal fashion 
with frequency /. The question posed is that of determin- 
ing the frequency of the external field such that B at the 
center of the slab rises to 90 per cent of its maximum 
possible value. 

I would like to digress here for a moment to remark 
that it appears to me to be frequently the case that one is 
asked to determine a constant in the boundary conditions, 
or a parameter of the problem, such that the solution will 
have a given form. This is often a way of measuring a 
physical constant ; indeed, when one finds a problem whose 
solution is sensitive to some parameter, then this may well 
provide a way of measuring that quantity with a high 
degree of precision. 

Returning to the problem, it is immediately noted that 
in heat flow or diffusion there is no concept of velocity of 
propagation; hence the ideas of characteristics and cones 
of disturbance are of little help. Nevertheless, there is a 
condition on the spacing in x and t. To arrive at a neces- 
sary condition, suppose that at some point an error ein H 
is committed, due, perhaps, to roundoff. This produces in 
turn an error of 2c in the second difference. Following this 
through it is found that there is an error of 
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in the estimate of -S^^^^^, since a difference equation of the 
form 



Bt''' = 5^'^ + 



c^'At 



47ry(A^)^ 



A'Hi 



is used. When the value of B is extrapolated to the point 
B^^^ for the next cycle the error becomes 

c^At 



4^iAxy ' 

Using this to calculate the new H of the next cycle, it is 
found, on expanding in a Taylor's series and keeping only 
two terms, 



f/r( )=Hr(5ro + 



c^At 



dH 



47ry(A^)' dB ' 

If the original error e is to produce an effect in the next 
cycle at the same point that is not greater than itself, then 
the following condition must be met, 

^.^4 ^{Axy dB 
^ = 3c2 dH ' 

This condition differs from that of hyperbolic equations 
in that it depends on the square of Ax. Thus, if the spac- 
ing in Ax is halved, the A^ must be divided by 4. This is 
typical of parabolic equations. The inequality takes care 
of a single roundoff, while if a roundoff at each point is 
assumed, an additional factor of 7/10 is needed on the 
right-hand side. 





H 



Figure; 3. Hystkrusis Loop, Molybde;num Pkrhalloy 



This condition is clearly necessary in order to expect 
anything lil«e a reasonable solution ; its sufficiency will be 
discussed later. Note the derivative dB/dH on the right- 
hand side. 

The particular sample of permalloy discussed had a 
B-H curve, as shown in Figure 3. Recalling the importance 
of the derivative dB/dH in the net spacing condition, it is 
seen that as the problem progresses a very tight spacing 
must be accepted throughout the problem or else the spacing 
at various stages must be modified to take advantage of 
those places where the derivative is large. The latter was 
chosen. 

In the early stages of the computation, while an attempt 
was made to obtain an idea of the order of magnitude of 
the frequency /, a crude subdivision of the slab into four 
zones marked by five points was made. By symmetry one 
half could be ignored so that, in fact, only three points 
needed to be followed. The outer point was driven by the 
boundary condition, while the two inner points followed 
according to the difference equations. 

To test the method, first a B-H curve was used which 
was a straight line. The comparison with the analytical 
solution was excellent. To show the reality of the net 
spacing condition the problem was deliberately allowed to 
run past a point where the net should have been tightened. 
The results are shown in Figure 4. This oscillation is typi- 
cal of what happens when a net spacing condition is vio- 
lated, although sometimes it takes the form of a sudden 
exponential growth instead. Indeed, when such phenomena 
occur, one may look for a violation of some net spacing 
condition. 

I have emphasized that the condition just derived is a 
necessary condition. There has been a lot of discussion 
lately as to whether this is a sufficient condition. Unfor- 
tunately, I do not have time here to go into this matter 
more fully. Instead, let me present some of the results 
obtained several years ago when we did this problem. 
Figures 5, 6, and 7 show a smooth transition on the 
solution as the frequency / was changed. Any errors are 
clearly systematic. The jumps in the inner points are due 
both to the shape of the B-H curve and the extremely 
coarse spacing of three points. When a finer spacing of 
five points was used (eight sections of the slab instead of 
four), much the same picture was found. The labor, of 
course, was eight times as much since there were twice as 
many points, and the A^ was decreased by a factor of 
four. This crude spacing should indicate how much valu- 
able information may be obtained from even the simplest 
calculations when coupled with a little imagination and 
insight into the computation. 

There seems to me to be no great difficulty in setting 
up such a problem for machine computation ; so I shall not 
go further except to note that in the original setup of the 
problem we provided for the fact that we would have to 
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consult not one but a family of B-H curves, the one chosen 
for each point depending on its maximum saturation. This 
refinement was not included in the results shown before, 
and in any case it produces only a small effect. 

The Trave;i,ing Wave Tube 

The last example I wish to consider is that of the travel- 
ing wave tube. A traveling wave tube consists of a helix 
of wire wound around an evacuated cylinder. The pitch 
of the helix reduces the effective velocity of the field due 
to an impressed electric current in the hehx to around 1/10 
that of light. Inside the helix is a beam of electrons going 
at a velocity slightly greater than the electromagnetic wave 
from the helix. As in the two-beam tube, the stream of 
electrons interacts with the field and gives up some of its 
energy to magnify the signal impressed on the helix. 

The equations describing one particular idealization of 
such a tube are 



dAjy) _ 
dy 

v(yl = 



dy 

_£_ 
dy 



'■ire I 
J 

WeAiy)J^ 



sin <f>(d, y)d6 

2v 



cos <l>{6,y)d$ 
q{6,y) ^A{y)sm<f>(d,y) 
<l>{e,y) ^k + -n{y) +2eq{d,y) . 



These equations have already been transformed over to a 
coordinate system moving with the wave. The 3; coordinate 
measures, in a sense, the length down the tube, while the 6 
measures the time. 

If the equations are examined more closely, it is seen 
that, for each 6, the lower two equations must be solved 
in order to move the solution of q and (f> one step down 
the tube in the y direction. The sine and cosine of <f> are 
then summed to obtain numbers depending on the funda- 
mental frequency. The higher harmonics are clearly being 
neglected. These upper equations in turn supply the co- 
efficients for the lower equations. This neglect of the 
higher harmonics was justified on the physical grounds 
that the helix damped them out very strongly. As a check, 
the amount of the second, third, fourth, and fifth har- 
monics in the beam was calculated later, and it was found 
that they could indeed be neglected. 

The first step is to make a transformation so that the 
parameters e and k drop out of the equations. 

Proceeding much as in the two-beam tube, it was de- 
cided that 16 points would provide an adequate picture in 
the 6 direction. Thus, there are 16 pairs of equations like 
the lower ones to solve. In addition, it was desired to 



solve eight such problems for different parameter values 
(which appear in the initial conditions and enter in the 
"linear" solution used to start the nonlinear problems). 
This gives 128 pairs of equations to be solved at each 
cycle, a situation very suitable for IBM equipment. On 
the other hand, the upper two equations only occur once 
per problem, or eight times in all, which makes them un- 
suitable for machine solution. Thus, the solution of the 
lower equations and calculation of the sums corresponding 
to the integrals of the upper equations were accomplished 
on IBM machines, while the rest of the upper equations 
were solved by hand calculations. Included in the hand 
calculations was a running check that may be found by 
integrating the equations over a period and finding the 
ordinary equations that govern the mean values. 

With a spacing chosen in one variable, how is the spac- 
ing to be chosen in the other? In this case, there is no 
theory of characteristics, in fact, very little mathematical 
theory available at all. The obvious was done. A number 
of spacings were tried, with a crude net space in A^, and 
the maximum permissible Ay, at that stage of the problem, 
was determined experimentally. Then a spacing Ay was 
chosen, comfortably below this limit, although not so far 
as to make too much work, and the calculation started 
with the hope that this would either be adequate for the 
entire problem or that the effect would show up in a no- 
ticeable manner in the computations. No obvious anomalies 
appeared ; so presumably the spacing of 1/10 unit in 3; was 
adequate. The net chosen was rectangular with every other 
y value being used to calculate the </> and q, while at the 
other values the A and rj were evaluated. This produces 
central difference formulas which are the most accurate. 
The old cycle in ^" and q~ was labeled by a — , the current 
values of ^° and >;° by a °, and the predicted values of ^+ 
and q+ by a. -{-. The new values of ^ + + and r;++ were 
labeled + + • 

To set up a system of difference equations corresponding 
to the lower equations: first, an estimate of g+ was ob- 
tained, then this was used to find a reliable value of <^+ ; 
finally, using this </>+, a better estimate of (/+ was obtained. 
The difference equations describing this situation are 



r = r + 



q- + 



( 



10 

10 



+ g- + 



sm <f>~ -\- sm </> 



^sm<^ j 

•) 



L = S sin </>^ 

M = 2 g^ sin </>+ 
N = :S,cos<t>^ . 



The difference equations corresponding to the upper equa- 
tions have not been shown, but it has been indicated that 
the solution of both equations was made to depend on 
three sums labeled L, M, and A^. 

To simplify matters in finding the sine and cosine of </),the 
units of measurement of angle were changed from radians 
to 1/1000 part of a circle. The trigonometric functions for 
such angles can be found by consulting tables in decigrades 
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at every fourth decigrade. The advantage of such a unit is 
that the integral part of the angle automatically gives the 
number of rotations, and the fractional part gives the value 
at which to enter the table. 

Consider the basic cycle of computation. It is obvious 
that the accounting machine-summary punch will be best 
adapted to the summing of the quantities leading to L, M, 
and N. This is the natural point to start a cycle, since the 
cards from the summary punch will have the minimum 
amount of information, leaving the rest of the space on the 
cards for future calculations. These cards will be called 
detail cards. 

Each detail card needs to be identified uniquely. To (io 
this the problem number, the card number which is its 6 
value, and the cycle number which is its y value, are given. 
The information that the detail cards must carry at this 
stage to describe the problem is the current values of <f)~ 
and q~. In addition, it is convenient to have the value of 
the sine of the old angle, sin (jr. 

The master, or rate, cards — the information for which 
comes from the hand calculations — must have identification 
consisting of the problem number and the cycle number, and 
the values of the two dependent variables A° and 7/°. The 
procedure is : 

1. Key punch the eight master cards and sort them 
into their appropriate places, a matter of one sort on one 
column. 

2. Multiply with crossfooting to obtain the quantity, 

A' 



,+ (estimate) 



10 



sm </>~ + q~ 



which is an estimate of the q at the new cycle. 

3. Another multiplier-crossfoot operation produces 



o ^o 



^ 10 



which is the new value of (f>+. Now the sine and cosine of (f> 
must be found. 

4. Sort on <f> for three digits, 

5. Collate in the table values of the trigonometric 
functions, 

6. and 7. Using the multiplier, linearly interpolate the 
values of sine and cosine of <^. Each may be obtained with a 
single pass through the multiplier, provided there are only 
five figures in the table values and three in their first dif- 
ferences. The algebraic signs may be picked up from the 
master cards and held up for the detail cards which fol- 
low, so that with a suitable complement punching circuit 
the value and its algebraic sign may be punched. 

8. Collate again to remove the table cards and at the 
same time put the table back in proper order. (Inciden- 
tally, the same control panel is used for both operations 
on the collator.) 



9. Resort the detail cards so that they are again in 
order, both as to the card number and the problem num- 
ber, a matter of a three-column sort. 

10. Multiply-crossfoot to obtain the new value of q 
from the formula 

A° 
q^ = q- + — (sin </>- -^ sin <^+) . 

11. Multiply to obtain q^ sin <f>*. 

12. List the calculated values, the three sums L, M, and 
A'^, and summary punch the cards for the next cycle. 

If these operations are gathered together, it is found 
that there are 6 passes through a 601 type multiplier, three 
sorts for a total of 7 columns, two passes through the col- 
lator, a key punching of 8 cards, and one pass through an 
accounting machine-summary punch for each cycle. 

We used our own accounting department machines with 
a 601 multiplier modified to have sign control, and three 
selectors. We operated only at times when they were not 
doing their main task of getting out the pay checks ! Need- 
less to say, no arguments ever arose as to priority on the 
use of the machines. 

CoNClvUSION 

Let me summarize the points I hope I have made. First 
and foremost, there is a large class of problems where the 
relative size of the net spacing chosen is of fundamental 
importance. Where there is no known mathematical the- 
ory, or where one is ignorant of it, one may still proceed 
on an experimental basis and watch for either violent 
oscillations or sudden exponential growths to indicate 
where the going is not safe. 

Second, it is not hard to set up a method of computation 
for a given problem, and one can estimate the accuracy at 
any step by some such device as using a Taylor's expansion 
with a remainder. The harder problem of propagation and 
compounding of errors I have not answered at all defi- 
nitely, but have suggested that prudence, physical intuition, 
and faith will provide one with a suitable guide. 

Lastly, it is not hard to work out the details of a basic 
cycle if one keeps in mind the amount of information that 
must be available at certain stages, watches the flow of in- 
formation, and has the courage to try to work out the 
details of a plan. When it comes to comparing alternate 
methods I presume that one can count operations, judge 
reliabilities, etc., of the various alternates. There may be 
better ways than you have thought of, but don't let that 
stop you! If the method is sound, economically worth 
while, then you are justified in going ahead. You don't 
need super computing machines, although they are nice to 
have ; you can go ahead with what you have at the moment 
and obtain useful and valuable results. 
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DISCUSSION 

Dr. Hammer: The choosing of networks in comparison 
to the interval of the variables sometimes can be avoided 
by using a different system of integration ; that is, an im- 
plicit calculation in which perhaps some of the variables 
are first found by explicit integration, and then they are 
recalculated. 

Dr. Hamming: You are thinking of the Riemann 
method, no doubt, or the von Neumann method of getting 
a difference equation which involves the present values 
and simply adjacent values one cycle forward. 

Dr. Ham^mer: Yes. One essentially calculates all the 
values at the same time, and then the condition you men- 
tioned can be violated to some extent. 

Dr. Her get: The graphical way in which you portray 
the effect of (At)^ to l\x is very good, and I think it has 
been stated in some of these meetings before that to be 
safe for the convergence involved in this process, A^ 
should be about half of A-*"- Isn't that right ? 

Dr. Hamming: You can't say any l\x and A^. It de- 
pends on the scale of the variables used. If the variable is 
multiplied by 10, the spacing would be changed numeri- 
cally. The condition is stated in terms of the velocity of 
propagation of signal in the hyperbolic case, and in the 
parabolic case one considers the derivative dB/dH. 

Dr. Grosch: I would like to ask a question about the 
nature of the oscillations encountered when the condition 
is violated. Have you made any effort to see, if you will 
pardon a nontechnical term, what the mean curve through 
those oscillations does? Does it follow the solution? 



Dr. Hamming: Yes, it does. 

Dr. Grosch: That is an interesting point. 

Dr. Ham.m,ing: If you examine Figure 4, you can see 
this is true. 

Dr. Grosch: We had a situation like that arise back in 
1946 when we were using 60rs, and in our case the con- 
dition on the very short A^ and Af was not a simple 
constant but a sort of variable of the column. We had this 
oscillation happen just a few x intervals from the end ; we 
tried a fudging method of this sort, and it seemed to work 
out all right. 

Dr. Alt: I think the situation concerning propagation of 
the local errors is not as hopeless as you indicate. One can 
use Green's function in order to study the propagation of 
errors whenever Green's function is available. If it is not 
in there, at least one can try to linearize it. We have tried 
that in a nonlinear problem, too, and it worked out. It can 
be done. I felt that the problem was simple enough that 
one didn't even consider publishing it, because it was just 
a straightforward application of the Green's function. 

Dr. Hatnming: I agree with what you say completely if 
your problem is either linear or your solution is reasonably 
close to linear with a perturbation, but when you encounter 
essentially nonlinear effects, Green's function will tie you 
up hopelessly when it is essentially the nonlinear part you 
want. That was the problem in all the examples that I 
showed; not to get the linear problem with the perturba- 
tion, but to get the essential nonlinear effects and see 
where they entered, how much they entered, and where 
they cut off. 
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THE USUAL METHODS for determining numeri- 
cal solutions of partial diflferential equations with specified 
boundary conditions are based on the approximation of 
the differential equation by a difference equation. In the 
case of the two-dimensional Laplace's equation, which 
is the only one I will speak of today, the differential 



equation is t-^ + -r~; 
ox- ay- 



0. The standard approximation is 



= where l^xf is the second difference 



Ay ■ N'yf 

in the x direction and Ay/ is the second difference in the 
3/ direction. 

The exact relation between the differential operator and 
the difference operator is an infinite series in the difference 
operator. 
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and the standard approximation amounts to cutting off 
this series after the first term in x and in y. If only second 
differences are to be used, it is obviously advisable that the 
interval A^ should be chosen sufficiently small so that 

1 AV 



the term 



becomes negligible. If this leads to too 



12(A^) = 

small an interval, we try to recover the accuracy by in- 
cluding more terms of the series in the approximation. 
The validity of this procedure needs rigorous justification, 
but it presents a practical computational approach. 

Now differences are linear relations between function 
values at adjacent points. Hence, any method which works 
basically on the difference equation will be a method deal- 
ing with values of the function at specified points within 
the boundaries. These points are generally chosen sys- 
tematically to cover the interior of the region with a regu- 
lar grid. We. shall consider only a square grid, as it is more 
easily adapted to machine computations than are the tri- 
angular or hexagonal grids. 

The direct approach to the problem is to write out the 
difference equation for each point of the grid. Since we 
are dealing with a square grid, A^ = A3', we shall call 



this mesh distance h. Then the difference equation at the 
point X = Xi, y = 3;^ will become : 

^ [f(xi-:i>yj) - ^fi^i^j}) + f{xui>yj) 

■j-f(xi,yj.^) - 2f{Xi,yj) -{- j{xi,yj^^)] = 
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There is one such equation for each point of the grid. 
Each equation may involve only interior points, or interior 
points and boundary points. If the boundary points are 
considered as known and transferred to the right sides of 
the equations in which they occur, then there is a system 
of w equations, one for each grid point, involving n un- 
knowns, the values of the function at each grid point, 
which completely defines the function at the grid points 
within the boundary. In the present case, it can easily be 
shown that a unique solution of these equations always 
exists. 

There is one great drawback to this approach to the 
numerical solution of a differential equation, and that is 
the number of equations in the system. Consider a rela- 
tively simple heat conduction problem. We have a cube, 
10 cm. on each edge, at a uniform temperature of 0°C. 
We place this cube in contact with a heat source along one 
face. The temperature of the source is some function of 
both coordinates. We insulate one of the adjacent faces 
and then inquire as to the distribution of temperature over 
the free faces ten seconds after contact is made. This 
problem will reduce to a four-dimensional case, three 
space dimensions and one time dimension. If a ten-point 
grid is introduced in each dimension, a system of 10,000 
equations in 10,000 unknowns results. And, while a large 
number of coefficients will be zero, this is not a problem 
to be approached with equanimity. Although this example 
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was designed to show how rapidly the number of equa- 
tions can increase, and is not the type of problem that 
would be solved in practice, problems of the same order 
of magnitude are available in the physically interesting 
problems whose solutions are being sought today. 

A second method of attack is the relaxation method of 
Sir Richard Southwell. Here one guesses at the value of 
the function at each grid point and then systematically 
improves the guess. The values of the function are substi- 
tuted into Laplace's difference equation, and the result 
will in general differ from zero. This difference, or 
residual, is computed for each grid point. The largest re- 
sidual in the entire field is now located, and the value of 
the function at that point altered in such a way that the 
residual becomes zero. This is equivalent to adding one 
quarter of the residual to the residual at each of the four 
adjacent points, leaving the rest of the field unaffected. 
The field is again scanned for the largest residual, and it 
is reduced to zero by changing the value of the function 
at that point. The process is continued until all the re- 
siduals become small, one or two units in the last place. 

As a hand computing method, relaxation has many ad- 
vantages. It deals with only a few points at a time, it in- 
volves very simple operations, and it converges to the 
solution of the difference equation rather rapidly. And 
there are variations — over-relaxing and under-relaxing, 
group relaxing and block relaxing — which increase the 
speed of convergence. As a machine method, many of 
these advantages are lost. The speed of convergence de- 
pends on relaxing the largest residual at each step. Hence, 
the entire residual field must be scanned before each oper- 
ation to locate this largest residual. This scanning for size 
is still a very inefficient operation, particularly when it is 
interposed between every set of five additions. Then, too, 
the block and group relaxations, which so speed up the 
convergence in hand computing, are very difficult to apply 
using automatic computing machinery. 

Another method related to the relaxation method is 
Liebmann's smoothing method. Once again, we start with 
the basic difference formula 

If now we multiply the equation by h^ and then transfer 
4f{xi,yj) to the right-hand side, we have an equation de- 
fining f(xi,yj) in terms of the four adjacent values of the 
function. The method consists in guessing the value of the 
function at each grid point, and then applying the smooth- 
ing formula to each point of the grid. The entire field is 
smoothed again and again until no changes are introduced 
in the function values to the degree of accuracy required. 
This method has some advantages and some disadvan- 
tages. Its main disadvantage is its slow speed of con- 
vergence. Its advantages are that it deals with only a few 



points at a time, that it involves only simple operations, 
and it is adaptable to machine computations. M. Karmes, 
of New York City, has done this, reporting in 1943 on an 
adaptation of this method to 601 multipliers. His machine 
method is straightforward, one quarter of the value of the 
function at the four neighboring points being summed to 
give the value at the central point. To assemble correctly 
the cards to be summed, Karmes prepares four decks of 
work cards. Each of these contain ^f{xi,yj), but they differ 
in that a second argument is introduced in each deck. One 
contains (i — 1,/) ; one, {i -\- 1,;') ; one, (i,j — 1) ; and 
one, {i,j -\- I). The four decks are now sorted together on 
the second argument and summed, summary punching the 
new value of the function at each grid point. The deck 
with the new function values is then reproduced four 
times, the second arguments are put in, and the process is 
repeated. This cycle is continued until the function values 
converge within the required accuracy. 

A method similar to Liebmann's method, but better 
adapted to machine computation, has been devised by 
Dr. Milne and tested on the 604 electronic calculators at 
the Institute for Numerical Analysis. Dr. Milne was seek- 
ing to avoid the sorting problem that led Karmes to the 
use of four decks of cards. He added two difference oper- 
ators, each satisfying Laplace's difference equation, to- 
gether. The first is the usual 
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Multiplying each by four and summing, we obtain 
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The trick now is to multiply through by /j^ and then add 
Z6f{xi,yj) to both sides of the equation. This gives 



= 2>6f{xi,yj). 
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This equation is now factorable, and we can define two 
operators U and V such that 

U fixuVj) = ^[/(^i-i,.Ty) + ^f(^i,yj) + f(^ui,yj)] 

yf('Vhyj) = -^[f(^i>y}-i) + 4/(;t;i,3;j) + /(^i,3'^i)]- 

If these are applied successively to the w*** approximation 
to the function values at the grid points, they will yield 
the n -\- P^ approximation. Or, 

rH^i,yj) = UVf(xuyj) = VUf-(xi,yj). 
The last equation indicates that the operators commute 
and that rows or columns may be smoothed first. 

This method works nicely on the 604 electronic calculat- 
ing punch. For each iteration, the cards must be fed 
through the machine twice, once in row sort and once in 
column sort. At the end of the second run, a new set of 
function values will have been computed for each grid 
point. 

The example tested a 9 x 10 rectangular grid with values 
of arctan x/y given along the boundaries. We have avail- 
able 10 place values of arctan x/y, so that a check was pos- 
sible on the speed of convergence. The smoothing was ap- 
plied first by rows and then by columns, although this 
choice was completely arbitrary. 

The wiring of the 604 control panel was simple and 
straightforward. The value of the function was read into 
factor storage 3 and 4, and a 4 and a 6 were emitted into 
the MQ and factor storage 2, respectively, on the read 
cycle. The analysis chart reads as shown below : 



The result of this operation is to punch V f(xi,yj.i) on the 
{i,j) card. The two last transfers set up the operation for 
the next point. This arrangement of storage units will 
handle any size numbers up to eight digits, and that should 
include all problems of practical interest today. There is 
no question of the function values growing too large, as 
the maximum and minimum values must occur on the 
boundary. 

The wiring of the 521 control panel is a little more com- 
plicated, as it was desirable to make the control panel auto- 
matically change itself for the dififerences between the first 
and second runs. There are two problems that must be 
handled on the 521 control panel. The first is that the input 
field for the second run is the same as the output field of 
the first run. And the second is the shift in argument. 

The card layout is as follows. In column 1, punch the 
row identification i. In column 2, punch the column iden- 
tification y. In columns 3-8, punch the original value of the 
function. After smoothing along a row, punch the answer 
in columns 9-14; and after the next smoothing along a 
column, punch the answer in columns 15-20. 

The first problem, then, is to read from columns 3-8 on 
the first run and punch into columns 9-14. On the second 
run, read from columns 9-14 and punch into columns 
15-20. This is done through punch selectors, the normal 
side being wired to the first run read and punch fields, and 
the transferred side being wired to the second run read 
and punch fields. The selectors are transferred by a y 
in column 80, a punch which is introduced on the first run 
by wiring from the emitter through the normal side of a 
punch selector to the punch magnet for column 80. The 
selectors which switch the read fields should be controlled 
through their immediate pickup, while the selectors that 
control the punch field should be controlled through their 
delayed pickup. 

The shift in argument is easily handled. On the first run, 
the / identification is gang punched backward into the fol- 
lowing card, while on the second run, the i identification 
is similarly gang punched back one card. Column 2 is 
wired from the second reading brushes through a normal 
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point of a punch selector to the punch magnet of column 
22. Column 1 is wired from the second reading brushes 
through a transferred point of the same punch selector to 
the punch magnet of column 21. This selector is then con- 
trolled through its delayed pickup by the y in column 80. 
Two points might be examined in a little greater detail. 
The first of these has to do with the smoothing of the 
boundary values during the first run. As the cards are 
going through in row sort, the first and last rows will be 
entirely boundary cards, and the values punched into these 
cards will be the smoothed boundary values rather than 
the true boundary values. This is necessary for the correct 
application of the formula, as a consideration of the func- 
tion at the point (1,1) will show. Suppose first that we 
do not smooth the first row. Then we will have available 
at the end of the first run these values on their correspond- 
ing cards : 

i j function 

1 /(0,1) 

1 1 ^[/(1,0)+ 4/(1,1) -h/(l,2)] 



2 1 



1 



[/(2,0)-F 4/(2,1) +/(2,2)] 



At the end of the second run, the answer punched in the 
card for the point (1,1) will be 

^ [6/(0,1) + 4/(1,0) + 16/(1,1) + 4/(1,2) 

-f/(2,0) -h 4/(2,1 )-H/(2,2)] 
which is equivalent to the true expression only if i{i,j) is 
linear along the boundary / = 0. 

If, on the other hand, we smooth the first row, we will 
have available at the end of the first run these values on 
their corresponding cards : 
i j function 

1 ^[/(0,0)-F 4/(0,1) -f/(0,2)] 



1 1 



2 1 



^[/(1,0) +4/(1,1) +/(1,2)] 



^[/(2,0)+ 4/(2,1) -h/(2,2)] 



At the end of the second run, the answer punched on the 
card for the point (1,1) will be the correct expression 

^ [/(0,0) + 4/(0,1) + /(0,2) -f- 4/(1,0) + 16/(1,1) 

+ 4/(1,2) -h /(2,0) + 4/(2,1) + /(2,2)] . 

Thus the use of a smoothed boundary value in the second 
run actually is necessary for the successful evaluation of 
the smoothing formula at all points of the grid. 

The second point is the use of a single delay in trans- 
ferring the selector, which governs the gang punching of 
the i identification on the second run. Standard practice 
for gang punching through a selector is to use a double 



delay so that the card containing the pickup punch will be 
passing the second reading station when the selector trans- 
fers. In this case, all cards have the pickup punch. Thus, 
use of a double delay would transfer the selector from the 
time the first card is passing the second reading station 
until the last card is passing the reading station. Use of 
a single delay will transfer the selector from the time the 
first card is passing the punching station until the time the 
last card is passing the punching station. Either type of 
delay will give the correct gang punching result in this 
case ; so a single delay was used as a simpler method. 

The complete sequence of operations now can be sum- 
marized. A deck of cards containing the boundary values 
is reproduced a large number of times. A deck of cards 
containing the initial values of the function at the interior 
points is key punched. This deck and one of the boundary 
decks are then sorted on columns 2 and 1. This puts the 
cards in order of column number within rows. The cards 
are then run through the 604 and again sorted on column 
22. This orders the cards on rows within columns. The 
first and last columns are removed, as they contain spuri- 
ous smoothed values. The remaining cards are again run 
through the 604 and then sorted on column 21. The first 
and last rows are removed, as they again contain spurious 
values. The remaining cards are reproduced, reproducing 
columns 21 and 22 into columns 1 and 2, and columns 
15-20 into columns 3-8. These new cards form the deck 
for the interior points in the next approximation. They 
are combined with a new boundary deck, and the process 
is repeated. 

For the example we tested, one cycle of operations on 
ninety cards took about five minutes. More time must be 
allowed for reproducing new boundary decks, but certainly 
ten steps an hour can be accomplished if a 604, reproducer, 
and sorter are set aside for the problem. And then an 
occasional check must be made of the convergence of the 
solution. We listed the ninety cards after every tenth itera- 
tion and examined two successive lists for changes. In 
about sixty iterations, we had reached an accuracy of 
about two units in the fourth place. 

This same example was tested in our hand computing 
section, using a mixture of smoothing and block relaxing. 
The field was smoothed three times, then block relaxed. 
This cycle was repeated three times, and then three addi- 
tional smoothings were made. At the end of these twelve 
smoothings and three block relaxings, answers were 
reached that were closer to the true answers than had been 
reached in the 60-odd iterations by punched card machine. 

The great difference in the speed of convergence is due 
to the use of block relaxing. An intuitive idea of the 
reason for this is gained by considering the basic action of 
the smoothing operator. Now Dr. Milne's smoothing oper- 
ator will work just as well on the residuals as on the func- 
tional values. The residuals are defined with respect to 
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this operator in a similar fashion to the way residuals 
were defined with respect to Liebmann's operator. Now, 
consider the original residual field and the effect of the 
smoothing operator on it. If the residual were plotted 
vertically against the x and 3; coordinates of the points and 
a surface passed through the ends of the residuals, a three- 
dimensional model similar to a mountain would be ob- 
tained. As the original guesses were not good, plus and 
minus errors would be found, large and small errors, and 
the mountain would be rough-covered with peaks and 
valleys. A few applications of the smoothing operator will 
level off the peaks and fill in the valleys, producing a 
smooth instead of a rough mountain. The outstanding de- 
viation from smoothness will come at the boundary, where 
the elevation of the mountain goes to zero. And beyond 
the boundary, a flat, level plane stretches to infinity in all 
directions. The task of the smoothing operator is to erase 
this lack of smoothness at the boundary by forcing the 
entire mountain out through the boundary. And as the 
altitude of the mountain decreases, the slope at the bound- 
ary approaches closer and closer to zero, and less and less 
of the residual is removed with each iteration. Hence, the 
convergence is rather poor, because the operator is most 
efficient at smoothing and inefficient at forcing residuals 
through the boundary. 



This situation is completely upset when block relaxing 
is added as a further tool. Now one smoothes for a while 
until a smooth mountain is formed. Then one traces a few 
approximate contour lines along the mountain. The area 
between any two contour lines is then dropped in altitude 
by the mean altitude of the two adjacent contour lines. 
This then removed the bulk of the mountain, leaving small 
peaks and ditches. These are rapidly smoothed over by 
use of the smoothing operator, and again the bulk of the 
mountain is carted off by the block relaxation. 

While the use of block relaxing together with smoothing 
provides a rapidly convergent way of solving Laplace's 
equation, it is at present not set up for machine computa- 
tion. The need for drawing contour lines and the inter- 
dependence of neighboring points makes it very difficult 
to set up for automatic calculation on present day calcu- 
lators. There are undoubtedly ways of accomplishing the 
same thing without using block relaxation in its standard 
manner, but these must be found by further investigations 
and offer problems which I hope Dr. Milne will inves- 
tigate during his next stay at the Institute for Numerical 
Analysis. 

DISCUSSION 

[This paper and the following one by Dr. Harry H. Hummel were 
discussed as a unit.] 



An Eigenvalue Problem of the Laplace Operator 
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IN A PAPER presented at the November, 1949, meet- 
ing at Endicott, Flanders and Shortley^ discussed the 
solution of the equation 

V-</' = a'/'- (1) 

Here a is an eigenvalue, and the problem is to find its high- 
est value and the corresponding fundamental eigen func- 
tion \p for homogeneous boundary conditions. This paper 
will describe the solution of this problem on IBM machines 
for a two-dimensional region consisting of a square with 
a square hole cut out of it (Figure 1). The function is set 
equal to zero at the outer and inner boundaries of the 
region. 

The solution of (1) is accomplished by transforming to 
a difference equation over a two-dimensional network of 
points in the usual way (Figure 1). Set \px+i,y + ^x-i,y + 
^oo,y+i + ^x.y-x ^ 5 ^x.y , whcrc \pa:.y is the valuc of the 
function at {x,y). Then the difference form of (1) is 
(2-4) ij^x.v 
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Figure 1. Layout of Cards 



Here h is {Xn+x — Xn) = (y»i+i — ym), the net spacing 
in the difference problem, a is the eigenvalue of the differ- 
ence problem, assumed equal to a. 

By defining 

and A = 1 H — j^ 

equation (2) becomes 

f^^x,v = ^^x,y ' (3) 

Since the highest value of a is desired, the highest value 
of X is also desired. The number of homogeneous equa- 
tions (3) is equal to the number N of points {x,y) in the 
network, which is, therefore, the number of eigenvectors 
of the equations (3). For this algebraic problem it is 
known^ that — 1 < A < 1, and also that the set of eigen- 
vectors is complete. A solution xf/n will, of course, consist 
of A^ numbers {l/x,y, one for each point of the network, and 
will correspond to an eigenvalue K- 

Then a first approximate solution function i{/ can be 
analyzed in terms of the eigenvectors i{/n 






(4) 



Operating with (w — a)/(l — a), where a is a real number 
such that — 1 ^ a < 1, 



w — a 



^ 



YjCr^i^^^n 



(5) 



Thus, it is seen that, by performing this operation a 
number of times with various values of a, the amplitudes 
of higher eigen functions may be reduced as much as de- 
sired relative to that of the fundamental mode ^\, the 
eigenvalue Aj, which is usually nearly 1. The value of 
(A — a)/(l — a) as a function of Aand ais shown in Figure2. 
Flanders and Shortley^ discuss the selection of values of a ; 
concluding that the greatest efficiency is achieved by choos- 
ing the a values as the roots of the Tschebyscheff Poly- 
nomial of order equal to the number of iterations it is 
desired to carry out. 
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RESULT OF OPERATION WITH -^ 



Figure 2 



It has been found convenient to apply this polynomial, 
assumed to be of even order with roots symmetrical 
about 0, as follows : 

Operate twice with the fundamental operator <a to 
obtain w^iff, and then form the linear combination 
[(w^ — «^)/(l — «^)]'A» thus using the roots ± a. Using this 
result, one again iterates twice with w and repeats the proc- 
ess for another value of a, etc., until the roots are all used. 
It is desirable to use the roots in an order that will prevent 
high frequency oscillations from building up excessively, 
as the number of digits carried in the iterations may be 
exceeded at some points. 

The remainder of this paper will be devoted to a dis- 
cussion of the application of the fundamental operator <o 
to a function ij/. The problem is to compute the average 
of the four nearest neighbors for each point of the net- 
work. This is done simultaneously for all points of the 
network, and the resulting set of values is the new func- 
tion wif/. The network of points is shown in Figure 1. It is 
necessary to cover only half the square because of sym- 
metry. 
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In performing this operation on the machines a card is 
provided for each point of the network. The two direc- 
tions have been labeled x and 3; as shown in Figure 1 ; the 
X and 3; identification numbers are punched on each card. 
To run the cards through a machine consecutively in the 
3; direction, they are sorted first on y, then x, and vice 
versa. 

A new deck is used for each iteration. The sequence of 
operations in an iteration is as follows : 

1. By a gang punching and reproducing operation on 
the reproducer with the cards running in the 3; direc- 
tion, the new function and its y neighbors are 
punched in the new deck from the old deck on which 
the new function has just been calculated. Both 
decks have been sorted on y, then x. 
New deck is sorted on 3;. 

Deck is run through the 602-A consecutively on x, 
allowing the x neighbors to be read from cards 
ahead and behind. Thus, the average of four neigh- 
bors can be calculated. 
Deck is sorted on x. 
Cards are listed to check for errors. 
New function is reproduced and gang punched into 
still another deck, starting another iteration. 

The following special cards are used and necessitate 
control wiring on the machines (Figure 1). 

5 or successor cards: For these the predecessor neighbor 
is eliminated. These occur next to the outer zero bound- 
ary, for which no card is provided. 

2P cards: For these cards, which occur on the diagonal 
line of symmetry, the predecessor is substituted for the 
successor, which is not in the deck. That is, for the 
point Xn, ym on the diagonal, on the 3; run one substitutes 
{^n, ym-i) ( =^«+i, ym) for Xn, ^m+i, and on the X run sub- 
stitutes (xn-i,ym) {=^n,ym+i) for ;»r„+i, 3;^, thus obtaining 
the proper neighbors. 

Blank cards: These are used for the zero boundaries of 
the hole. Punching is suppressed for them on the 602-A 
so that they provide zero neighbors for adjacent points of 
the network. 

The operations in the reproducer run are shown in Fig- 
ure 3. Note that the cards in the old deck must run one 
ahead of those in the new one. It is desirable to have aux- 
iliary identification on one deck or the other so that iden- 
tifications may be compared. The cards shown are at the 
end of one row and the beginning of another, illustrating 
the operation of the 2P and 5 controls in the 3; direction. 

The programming of the 602-A is shown in Figure 4, 
and control panel wiring is shown in Figure 5, page 32. 
Here is formed 0.25 times the sum of the neighbors (de- 
noted as 5), 
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The simplest means of calculating the eigenvalue A is 
simply to take the new total of the function for all points 
over the old total. This gives A = /wi/r/Ji/^. A more accurate 
value of A for the fundamental when higher modes are 
present may be obtained by forming /'/'('"'A)//"/'^- 

Special IBM techniques described in this paper were de- 
veloped by Mr. James Alexander of the Argonne National 
Laboratory. 
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DISCUSSION 

Professor Kunz: I would like to point out that there is 
some difference between this and Laplace's equation, in 
this sense: In Laplace's equation we have two items of 
error: (1) How close do we approximate the differential 
equation? And (2), how close are we to the solution of 
the difference equation ? We have those two, plus the fact 
that the definition of A from the difference equation is not 
the definition of A that is in the differential equation. This 
may be seen by considering the simple case of the vibra- 
tion, let's say, of a drumhead that is one by one. The 
eigenvalue in this case is 2Tr^, The appropriate finite dif- 
ference equations can be solved exactly. In fact, for any 
number of points taken, even though the number of in- 
terior points is only four or nine, the distribution is the 
same ; the actual distribution. It is a sine sine distribution 
of the drumhead. So no error is made in approximating 
the differential equation by a difference equation, as far as 
the characteristic function is concerned. But the definition 
of A is now AV/^V' which is not the proper definition in 
terms of the differential equation. 

I might mention in that connection that you can obtain 
a much better result by not iterating further or taking 
more points, but using simply a higher approximation to 
the Laplacian. 

Dr. Hummel: Yes; you do obtain a more accurate an- 
swer. 

Mr. Turner: The difBculties in convergence in Dr. 
Yowell's method and Dr. Hummel's method are associated. 
As was pointed out, the characteristic solutions of the 
equation V^ "A + ^ "A form a complete system. Suppose the 
equation were V^ ^ = and in carrying out the numerical 
operations at some point ij, instead of getting zero, we had 
cij. Then, I think, it is quite obvious that these ey's or the 
errors can be composed of linear combinations of the eigen- 
vectors associated with this problem. There are just as 
many c's as there are points, and there are just as many 
eigenvectors as there are points in your numerical differ- 
ence equation. 



Therefore, when a set of errors, a set of residuals, 
occurs, which forms a repetitive pattern, it turns out that 
they are actually composed of a combination either domi- 
nated by one particular eigenvector or made up of a linear 
combination of the eigenvectors corresponding to one 
particular eigenvalue. 

If we were to go through the operations (I have had it 
happen in actual numerical calculations that I get a set of 
residuals) and after operating on them, all we did was to 
change the magnitude of the residuals but didn't succeed 
in changing their distribution, then in that case it corre- 
sponds rather clearly. If we treat the set of simultaneous 
equations — that is, the matrix coefficients — as purely a 
matrix operator, then what we have done is carry out the 
operation A e = A e. That is practically the same thing as 
happened in Dr. Yowell's paper. 

If we will carry out this very simple operation we may, 
in one step, eliminate the dominant phase of these errors. 
First, at each point we form the sum of the absolute 
values of the errors. We also take each of the errors, each 
of the residuals, and operate on it with our matrix of 
coefficients. 

In other words, with the errors treated as the initial 
function yp, let's suppose that the result of operation equals 
X^. Let's call it some quantity v. Now, if we will form a 
second sum ; let's call this one A-B, which is equal to 
2(v|e|/e). This sum is to permit us to determine whether a 
particular eigenvector, which has perhaps both positive 
and negative signs, is dominant. Then an approximate 
value of the A is B/A, because in the operation on e to 
obtain Ae, that is ^ on e to get v, we have multiplied it by 
the latent root of the matrix of coefficients, which happens 
to correspond to the dominant eigenvector or combination 
of eigenvectors having nearly the same latent root. Once 
we have found A we can correct our original \p. If we now 
have errors of any substantial amount, corresponding to 
the higher eigenvalues, this will produce a roughness cor- 
responding to a small variation from point to point which 
the subsequent smoothing process will eliminate quite 
rapidly. 

Mr. Kelly: Has anyone had any experience along these 
lines of staying within the forced considerations of your 
mesh? Has anyone observed any forced oscillations of the 
type Dr. Hamming has found? We observed it staying 
within the mesh by a factor of 10 to 1, and still observing 
forced oscillations. 

Dr. Hummel: This business of oscillation depends on 
the range of the eigenvalues, doesn't it? If you know what 
the range of the eigenvalues is, you can certainly choose 
the mesh in such a way that you won't get the oscillation. 
You can always, of course, change your variables in such 
a way that you don't get the oscillations. This has been 
found true at least for the solution of the diffusion equation. 
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Dr. Alt: I have a question for Dr. Hummel in connec- 
tion with the process of speeding up your convergence by 
this trick. The division by (1— a) is not essential; that is 
just to bring the eigenvalues back into scale. But subtract- 
ing the constant a reminds me of something that I have 
seen in the literature that I am not sure is the same thing. 
It is in a paper by Aitken about 1937. It is the last para- 
graph of a very long paper, and is easily overlooked. What 
he mentions is this: Suppose -^ is a matrix and jr is a 
vector, and that you are trying to solve the equation 
Ax — A^ = for the largest value of A. If you replace the 
matrix A by A — a this matrix has the eigenvalue (A — a) . 
Some of the methods for finding the A's converge with a 
speed which depends on the ratio of the largest to the 
second largest A. We are trying to choose a so as to maxi- 
mize that ratio. But, as you mentioned, you have to make 
sure that some of the smaller a's don't become large in that 
process. 

I did not hear what you said about getting around that ; 
but there is an answer given by Aitken. Suppose your 
eigenvalues are A^, Ag, . . . , A„, and suppose they are all real 
and arranged in size here. What you want to subtract is 
the mean of the second largest and the smallest eigenvalue. 
It is important to choose it this way. After this, the second 
root and the smallest root become equal in size and oppo- 
site in sign, and the ratio of Ai to either of those is maxi- 
mized. When the eigenvalues are complex, it is a little 
more complicated. But you can see geometrically what 
point you have to choose for a. 

Dr. Hummel: That is essentially what I have in mind. 



Dr. Alt: There is a very brief mention of this simple 
method by Aitken in Proceedings Royal Society of Edin- 
burgh, 1936-S7. 

Professor Kuns: I would like to point out an even 
earlier work. There is an article by R. G. D. Richardson 
in 1911, which is one of the earliest works on stresses in 
a masonry dam. He considers the choice of a in quite some 
detail. It has been disapproved of by those who did not 
know of it. 

Dr. Ham,ming: This point was discussed quite exten- 
sively at the last meeting. Flanders went a little bit further 
in his discussions than has been, indicated here. 

In the first place, you are not restricted to a linear ex- 
pression. You can resort to polynomial expressions. What 
they apparently had done — Flanders, Shortley, and others 
— was to use the Legendre polynomials, which, as you 
know, have many roots spread out fairly low and rise 
sharply at the ends near 1, so that it multiplies this factor 
and keeps all the rest of the bounds down. In private con- 
versation afterward, we pointed out that he should not 
have used Legendre polynomials but should have used 
Tchebyscheff's equal ripple polynomials. In that fashion 
you are not restricted to this. You simply form a poly- 
nomial combination of about the order you want, which 
would be a Tchebyscheff polynomial — to keep the function 
down over the whole range, to keep all these down while 
working only on your maximum — and the degree has to be 
restricted so that your eigenvalue is not caught over the 
first zero of the polynomial which you are using. 
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THE PROBLEMS of engineering in which such sys- 
tems are found and which are successfully solved are: 

Determination of natural modes of free oscillations of 
structures.^ 

Determination of stresses in indeterminate structures.^ 

In general these problems cover all the variations of an 
actual structure ; therefore, the classical solutions are im- 
practical for the equations at hand. 

There is only one variation in this solution between one 
class of problems and the other; so the method will be 
briefly sketched for the first class (oscillations) for which 
it is more extensive. Let the differential equation be 

dy 

where the A's are functions of ;i: and may contain a char- 
acteristic number A, or w^. 

The method, which is presented in detail in reference 1 
and reference 2, is briefly described here. The problem in- 
cludes n boundary conditions and when one boundary con- 
dition is used in the differential equation, another equa- 
tion, called a secondary boundary condition, is obtained. 
There are n such equations. Altogether there are 2n 
boundary conditions. 

The unknown y is then written 






+ A^{x)^ + A,{x)y = 



y 



^Y^CiY, {x) 



where Ci are factors to be determined and Yi {x) are 
polynomials in .*• which satisfy the n boundary conditions 
and the n secondary boundary conditions. 
These polynomials Yi {x) have a form 



5i(0/^+^+... + 5,„,, 



ai-\-p{n-]rl) 



.1 (i) ^ 

where a^ and p are selected in a simple manner, and the 
B(i)'s are obtained from recurrence formulae obtained 
from the 2n boundary conditions. In general Oi and p are 
the same for the same type of problem, and the B{i)'s 
change very little with the coefficients of the differential 
equation. The polynomials for the many cases of bending 
and torsion oscillations are all to be found in reference 1. 



If this approximate y and its successive derivatives are 
used in the differential equation, a function e(x) is ob- 
tained which represents the error, as a correct 3; would 
make the left-hand side vanish. 

The two boundaries are called x^ and X2, and one equa- 
tion for the solution of the Ci's is obtained from 






:{x) Yi (x) dx = 0. 



There are j such equations, and they are homogeneous 
in Ci. They can be solved for i- — 1 of them as functions 
of the .y'th, provided that the determinant of the coeffi- 
cients of the Ci's vanishes. This condition provides an 
equation of the .y power of A, the j roots of which are 
positive. 

For any root Aj there results a set of coefficients Ci and 
therefore an approximate solution yi of the problem, or 
mode of oscillation. 

This method has many advantages which cannot be 
pointed out here. It can easily be set up for tabular or 
IBM calculations. When the An (x) are random curves, 
the integrations can be rapidly made by increments on the 
IBM machines, thus making the method very general. 

Its accuracy is very satisfactory: For instance, it is only 
necessary to make .$■ = 3 in order to obtain the first two 
modes y^ and y^ with an accuracy consistent with the engi- 
neering problem. 

It can also be said that the preceding integral equations 
happen also to satisfy the equation of least work for this 
first class of problems. 

On account of its simple algebraic form it is possible, 
as shown in Sections 1-T and 1-B of reference 1, to solve 
in advance the problem for a large family of cases. 

Other problems of structures which have the same type 
of differential equations but where the characteristic num- 
ber A has another meaning can be similarly solved. 

Thi; second CIvASS covers two-dimensional probleins of 
indeterminate structures. An important one is the determi- 
nation of stresses in box structures as in reference 2. The 
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key of this problem is the solution of a set of n linear 
differential equations such as 

with m = 1,2, . . . , n. The ^, B, and C are known func- 
tions of ;r, and the D's are known functions of ^ and of 
other independent variables. 

This problem has 2n boundary conditions, 2 per equa- 
tion. Moreover, when these conditions are used in the dif- 
ferential equations written at the boundaries x-^ and X2, 
2n secondary boundary conditions are obtained. This 
makes a total of 4n boundary conditions. 

The unknown ym is approximated by 

s 
ym = Yf {X) + VCr Ff {X) 

as in the first class of problems. 

The polynomials F^ {x) are made to satisfy completely 
the boundary conditions. Their form is 

F^ {x) = A^ -^ B^ X -\- C^ x"" + Df x\ 
The coefficients A, B, C, anc? D turn out to be easily com- 
puted as linear functions of the independent variables 
present in the previous Dm {x), and of the boundary 
values. 

The polynomials Yf {x) are set to satisfy the An bound- 
ary equations obtained when the right-hand terms made of 
independent variables and boundary values are made equal 
to zero. Their form is 



y- = ^j» ,/.+?+«■» ^.".+2/' 



where ai and p are easy to determine and the A, B, C, and 
D are found from recurrence formulae obtained from the 
An mentioned equations. 

For the box structure problem of reference (2) the func- 
tions Y'"'' {i = Q, 1,2,. . . ,s) can be found completely de- 
termined in this reference. 

The coefficients Cf in y^, are determined as in the first 
class of problems. However, the ns integral equations ob- 
tained for the determination of the same number of Cf co- 



J^i 



efficients are not homogeneous, and their direct solution 
gives these coefficients as linear forms of the independent 
variables of the problem. 

Considerable simplification is obtained by the use of an 
auxiliary variable s which varies from to 1 between 
boundaries. 

For the problem of reference 2 it was found with sets of 
six linear differential equations that the solution checked 
closely the solution based on consideration of the minimum 
square errors obtained by using the integrals 

Xo, , 

dim 
Cot {x) ^^ dx = 

Xi ■ ' 

although dim/dC^ is different from Yi^(x). 

Moreover, these solutions check well the solutions ob- 
tained by classical methods for sets of 6 linear equations 
with constant coefficients, although s was taken as 1 which 
represents the simplest solution. 

In general it will be found that these solutions, in addi- 
tion to their satisfactory accuracy, are several times shorter 
than the classical solutions, even for the simplest cases of 
such systems. 

The general solution of a given problem can often be 
carried algebraically up to the integrated form of the C 
equations, thus giving a compact solution which can be 
carried out simultaneously on IBM machines for several 
cases of the same problem in the time required for one case. 

Finally, the writer believes that the same method could 
be successfully extended to linear partial differential equa- 
tions, although he knows of no such application to have 
been made to date. 

1. North American Aviation Report NA-5811, "Method of deter- 
mination of the frequencies, deflection curves and stresses of 
the first three principal oscillations in torsion and bending of 
aircraft tapered structures." 

2. North American Aviation Report NA-48-210, "Determination 
of stress distribution and rigidity of multi-cell box structures." 

3. Aero. Res. Committee Reports and Memoranda: 

R & M 1799-Approximation to Functions and to the Solutions 

of Differential Equations. 
R & M 1798-Galerkin's Method in Mechanics and Differential 

Equations. 
R & M 1848-The Principles of the Galerkin's Method. 
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WE SHALL START with a set of simultaneous 
linear equations as being something familiar, and I shall 
restrict my discussion to three equations in three unknowns 
x-^, x^, and Xr^. The generalization to larger sets should be 
clear. We can write the coefficients of these unknowns by 
using a single letter a with two subscripts ; thus 

^\\^\ I ^12-^2 T" ^13-*'3 — ^1 

^21-*^l ~r ^22-'^2 "I <^23-*^3 =^ ^2 V ^ / 

^Z\^X \ *32-*^2 ~r ^33-'^3 — ^3 • 

Because of the presence of the constants h-^, b^, and ^g (at 
least one of these is assumed dififerent from zero), this set 
of equations is said to be nonhomogeneous. 
If the determinant of (1) 



A 



^0 



(2) 



then a solution exists and by Cramer's rule is 

^1 = ^(^1^11 + M21 + M31) 

^2 = ^(^1^12 + M22 + M32) } (3) 

^a = ^(^1^13 + M23 + M33) 

Here Ay is the cofactor of a^, i.e., 

A,,= (-iy^^^ij _ (4) 

where e/^^, the minor of a^y, is the determinant obtained 
from A, by crossing out the row and the column in which 
aij occurs ; thus 



-<1! 


n. 
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Matrices and Matrix Products 

Introducing the concept of a matrix and of a matrix 
product, one can write ( 1 ) in the form 

'^11 ^12 ^13 



or simply as 




= h. 



ax = b 



(6) 



where a, x, and b are termed matrices and stand, respec- 
tively, for the sets of numbers included in the parentheses 
of (5). A matrix is conceived of as a complex of all the 
numbers in the parentheses ; thus, if any of these numbers 
is changed, the matrix is changed. 

Two matrices are equal only if they have the same num- 
ber of rows, the same number of columns, and correspond- 
ing elements (numbers) are equal. Matrices of the type of 
X and b are called column matrices or vectors. In order 
that (5) be equivalent to (1), the product of the matrix a 
and the vector x must be the vector 

(^IS.^1 1 ^12-^2 ~r ^13-^3 \ 
^21-*'l I ^22-^2 I '^23''^3 I 
^31''^1 n~ ^32''^2 I %3-*^3 / • 

This may be summarized by saying that the elements of the 
product are 



{ax)i = ^ 



^ifc-^fc 



(7) 



fe=i 



In the same manner, the solutions .r^, x^, x^ given by (3) 
can be expressed by the matrix equation 

X = ar^b , (8) 

where the square matrix a~^ has elements 






(9) 



The matrix a~^ is called the inverse of the matrix a. The 
numerical determination of the elements of the inverse of 
a given matrix is one of the important problems of nu- 
merical analysis. 

Another basic problem is the finding of the product of 
two matrices when the number of rows and/or columns is 
large. The product of a matrix i4 by a matrix B can be 
taken only if the number of columns of A, say p, is equal 
to the number of rows of B. If this condition is met, the 
product is a matrix C having the same number of rows 
as A, say m, and the same number of columns as B, say n. 
The elements of C are 






a - 7 , 



AiuB 



ikOjcj , 



where i = 1,2, . . . , m and / = 1, 2, 
is a special case, n = 1, of this rule. 



(10) 
n. Clearly (7) 
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A matrix having n rows and n columns is termed a square 
matrix of degree n. It is easily verified that the product of a 
square matrix and a vector involves n'^ multiplications, and 
the product of two square matrices of nth degree requires n^ 
multiplications. 

NuMERiCAi, Solution oi^ Simui,taneous 
Lini;ar Equations and Matrix Inversion 

The numerical solution of a set of simultaneous linear 
equations by means of (3) is usually thoroughly imprac- 
tical when the number of equations n is of the order of ten 
or more. This is due to the excessively large number of 
multiplications required, namely, (n + 1) ! y{n), where 
1 ^ y(n) < e - \, (e = 2.718 . . .). For w = 10, this 
means about 70,000,000 multiplications. Compare this num- 
ber with about 1/3 n^ multiplications, or actually about 410 
when n = 10, needed for the elimination method. 

It is convenient in dealing with simultaneous linear 
equations to work with a matrix composed of all the given 
constants, 



(11) 



the so-called augmented matrix. From a computational 
point of view this set of numbers represents the original 
equations written with detached coefficients, much as is 
done in evaluating a polynomial by synthetic division. 
Equality marks can be imagined before each of the elements 
of the last column. 

It is clear from this point of view that the augmented 
matrix, obtained from a given matrix by subtracting from 
the elements of any row some multiple of the corresponding 
elements of a second row, is equivalent to the given matrix. 
By equivalent is meant that the solutions of the set of 
equations represented by these two augmented matrices are 
the same. 

Elimination Method 

In the elimination method l/a^ is computed, and all the 
elements of the first row are multiplied by this number.* 
The first element of this row is now unity; by subtracting 
from each element of each of the other rows the proper 
multiple of the corresponding elements of the first row, one 
obtains the matrix. 
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b, 
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K 



\0 flga 033 K I 
If the equation represented by the first row is dropped 
momentarily from consideration, two equations in two un- 
knowns remain. Therefore, we can deal with these elements 

^Actually, one needs first to arrange the equations in such a way 
that ail is one of the larger diagonal terms so that multiplying by 
1/aii does not reduce the number of significant figures. 



in the same way, thereby obtaining a 1 in place of the 
^22 and a zero in place of the a^. It is possible, then, to 
drop consideration of the second row. It is clear that by 
continuing this process the matrix can be reduced to the 
form 

(1 ^12 ai3 ^1 \ 
1 -a,, h (12) 

1 hj, 
in which the dij are equal to 1 along the diagonal and zero 
below. We will refer to (12) as the triangular form. 

Starting with n equations in n unknowns, this reduction 
will require n divisions and (1/6)m {n-\- 1) (2w +1) multi- 
plications. This assumes, of course, that there are no 
ones or zeros in the equations. For large n, there are 
about (l/3)w^ multiplications. The n divisions are negli- 
gible. 

Having obtained the triangular form (12), the last row 
may be multiplied by d^^ and subtracted from the second 
row, and by a^g and subtracted from the first row. This will 
cause zeros to appear where ajg and agg are in (12). Simi- 
larly by multiplying the new second row by d[2, the new d^^ 
obtained from the above process, a zero is established at 
that location. The equations are now in the diagonal form 

/ 1 hl\ 

I 1 &2* (1^) 

\0 \ hi/, 

where the stars simply indicate new constants obtained by 
the process. The solution of these equations and therefore 
of the given equations is x^ = h\, x^ = b*, and x^ = bl- 

The number of multiplications needed to go from (12) 
to (13) is only (l/2)« (w — 1) or, for large n, approxi- 
mately (l/2)«2. Thus, nearly all the computation is needed 
to obtain the equations in triangular form. In fact, for large 
n, the number of multiplications needed for the whole elimi- 
nation method is still approximately (l/3)«^. 

While considering this method, it is convenient to see 
how it may be carried over to the inversion of matrices. 
The technique is very much the same, but instead of work- 
ing with the augmented matrix (11), in which the b's are 
known numbers that can be multiplied, subtracted, etc., the 
b's are treated in the same way as the x's and each one is 
provided with a separate column. The augmented matrix 
now appears as follows : 

ffi3 1 0\ 

«23 1 ) (14) 

ass 1/. 

The numbers in the first three columns again represent 
the coefficients of x-^, x^, and x^, respectively, while the 
numbers in the last three columns represent the coefficients 
of &i, b^, and b^. Equality marks can be imagined between 
the third and fourth columns. Thus, one can look upon 
(14) as the equations (1) in skeleton form. 
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Without proving it, if the nine a's are operated upon in 
exactly the same way as before, by means of operations on 
individual rows of (14), letting the numbers accumulate as 
they will in the b columns, the following matrix is obtained 

\ 
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«12 


«13 


Cx 
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«23 


c. 








1 


c. 







(15) 



At this stage the matrices of the coefificients of the x'& and 
of the &'s are each in triangular form. The number of multi- 
plications needed for this step is about (l/2)«^. 

Now multiply the second row by a^^ ^^^ subtract it from 
the first row, thereby introducing a zero at the location of 
ai2. Similarly, by working with the third row, zeros can be 
obtained at the ajg and d^r^ positions. By continuing this 
process, the matrix can be written in the form 

/I c* .r, r*\ 

10 cl, cl, r* (16) 

\0 1 cl, cl, c:J ' 
The number of multiplications needed for this step is again 
about (l/2)w^ 

The three equations represented by ( 16) express Ji'i, Xo, 
and jTg, respectively, in terms of the &'s. Since the ctj are 
the coefficients of the &'s in these expressions, they are the 
elements of the inverse matrix [see equation (8)]. The 
amount of computation needed to invert a matrix in this 
manner is indicated roughly by the n^ multiplications, 
needed for the two steps above. The n divisions and many 
additions required are customarily neglected. 

Having the inverse of the matrix a, the vector h can be 
multiplied by it to obtain the numerical values of the x's. 
As seen earlier, this requires n^ multiplications. To solve 
k sets of equations with the same coefficients a^, but with 
different hi requires about n^ + kn multiplications. On the 
other hand, to solve the sets of equations individually would 
require about {k/?>)n^ multiplications. 

Therefore, if you have at least four sets of equations to 
solve, with the same ay but different hi, and the number of 
equations in the sets is twelve or more, it is better to first 
invert the matrix a. Also, it is necessary to invert the matrix 
if the &'s are not specified numerically or are treated as 
variables. 

Now, I would like to pass to other methods of finding 
the solution of a set of linear equations. As we have seen, 
the change in procedure from this, to the finding of an in- 
verse of a matrix, is not very difficult. One simply makes 
provision for keeping the h's separated, instead of allowing 
them to add together as one proceeds. 

There are a great many methods which are closely re- 
lated to the elimination method. Although some of these 
are very good for certain purposes, such as the Crout 
method for a desk-type calculator, I shall lump them with 
the elimination method. The methods I shall treat are 
chosen mainly for their interest. 



Square Root Method 

The square root method is not completely general. It as- 
sumes that the matrix a of the coefficients of the ;r's is 
symmetric. By this is meant that any coefficient a/i is al- 
ways equal to rty. This limitation on the form of a is not as 
serious as one might suppose, since many of the matrices 
arising from physical problems are symmetric. 

In this method, rather than obtain the intermediate tri- 
angular form (12) by some operations on the rows of the 
augmented matrix, we assume it and determine the condi- 
tions that such an assumption imposes. In particular, we 
assume that the matrix a can be expressed as the product 
of a triangular matrix S and its transpose S' , a matrix ob- 
tained from S by interchanging rows and columns. Or, 
more precisely (17) 








^22 



1 "m 0'2n ^3» • • • O'nnI Vi» "^2»t "^3« • • • "^i 

which is just the equation a = S'S. 

Therefore, in place of ( 5 ) we write 
'.Tn ...0 \ lk\ Ih^ 

J,, ^22 ...0 \ Ik 

i ^2Z "^33 • • • U j I " 1 I 

where the k matrix is the matrix product, 



10 J, 








(18) 
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(19) 



The elements Sij are obtained from the known elements 



of a by the use of the equations 



a, 



"1} 



(20) 



I T:rr- 

Su= ^ an — /^.yri , «' ^ 2 

Sij = — ( di} — y^^H Srj j , / > »■ ^ 2 . 

These equations result from multiplying the matrices 5*' 
and S in (17) and equating corresponding elements on the 
two sides of that equation. 

Having the Sij, (18) can be solved, very easily, for the ki 
and (19) for the Xi, since these equations are in diagonal 
form. These steps require only about 2w- multiplications. 
The total number of multiplications for the whole process 
is approximately (l/6)«^. 
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This method requires, therefore, only about half as many 
operations as the elimination method ; however, it is appH- 
cable only to symmetric matrices. While the taking of a 
square root is not usually as simple as a multiplication or a 
division, the process requires only n square roots and n 
divisions, which are negligible for sizeable n. The coding of 
this for automatic calculators has been tried, I believe. 
(Some difiQculty in this respect arises from the need, at 
times, to take a square root of a negative number, thus 
introducing i = ^/ — 1 into our computations. The num- 
bers, however, are either real or pure imaginary.) 

Iterative Methods 

The next method I would like to consider is an iterative 
method, a method of successive approximation, or some- 
times called the Gauss-Seidel method. In discussing this 
method I should like to treat a particular example. Again, 
I shall restrict myself to three equations in three unknowns. 
The particular equations I shall consider are : 
25:^1 + 2.ar2 -\- JTg = 69 "j 
2;iri -1^ 10.;r2 -f- .sr3= 63 V (21) 

^1 +. A"2 + 4;r3 = 43 ) . 

I have used a mathematician's prerogative to deal with 
nice round numbers. Seldom will you be called upon to 
work with such convenient numbers. 

Often new methods are developed by starting with very 
bold approximations. Here we shall initially assume that 
all the coefficients, that are less than 4, are small enough to 
be neglected. This makes it possible to write down at once 
as a zeroth approximation, 

25x1°) - 69 ) 



10.^'(«) = 63 



(22) 

4a'(«) - 43 ) • 

The equations (21) are approximated, thus, by a set of 
equations in diagonal form. 

This brings up a problem in terminology. Does one start 
with a zeroth or a first approximation? Well, I have 
adopted the following answer. You normally employ the 
designation first approximation, except when making a 
wild guess that cannot properly be justified; then it is a 
zeroth approximation. Surely in this case it is a zeroth 
approximation, which shall be designated by the matrix 
/ x^,'^ \ I 2.76 \ 

x''' = x^,^^ = 6.3 (23) 

\ x^i^ I \ 10.75 / . 

For purposes of comparison, the correct answer is 

X = 1 5 j (24) 

Surprisingly enough, the zeroth approximation gives the 
right order of magnitude for the x's. 

Now, having some idea of the size of the individual x's, 
v/e can go back to (21) and correct for the off -diagonal 
terms. Thus, the first approximation is written: 



2Sx^^^ = 69 - 2x(.p - ;»;(»> ") 

\Ox<:p = 63 - 2x{^^ - ;i-f > > (25) 

4,r<3i> = 43 - x^^^ - .^'(«) ) • 

The right-hand sides are known; so x'-'^^ can be found, 
which turns out to be 



x 



(1) — 




This still diflfers considerably from the answer given in 
(24), but progress is being made. 

Having a better answer for x, we are in position to make 
a still better estimate of the correction terms in (25) ; 
therefore, we can obtain a better approximation x^'^\ This 
process can be repeated as often as desired. Thus, in 
general 



25.ri**i> = 69 - 2x^2*^ - .r<« 
lOx^i'^^ = 63 - 2x{'^ - xi^^ 



(26) 



Ax'-t^^ = 43 - 


xi'' - 


and these equations can be written 


/;,(;.x)\ / 2.76\ 
;ir<*+i) = ^(i+i) j = ( 6.3 




/ -0.08 
+ (-0.2 

\-0.25 -0.25 


-0.1 




X)- 



(i) 



■ d) 



X\ 



(i) 



(27) 



■ d) 



At the sixth approximation, we have 

/ 2.002 \ 
x(«) = ( 5.0004 ) 
\ 9.0006/. 

These values are close enough so that we do not need to 
apologize for them, and, clearly, the iteration can be con- 
tinued with consequent improvement of the results, as long 
as desired. This is a Gauss-Seidel process. Whether the 
process converges depends on how large the diagonal terms 
are, compared to the oflf-diagonal terms. 

A slightly different technique may also be employed. In- 
stead of pruning off all the non-diagonal terms, just those 
terms above (or below) the diagonal may be removed. This 
reduces the equations for the zeroth approximation to tri- 
angular form. To solve these, only m^ multiplications are 
needed instead of the (l/3)w^ needed for the original 
equations. 

This method is due to Morris. He showed that if the 
matrix of the coefficients of a is positive semidefinite, 
which means that the associated quadratic form is either 
positive or zero, then the process converges. The iteration 
equations here are 

25x(ri> = 69 - 2xf - xf \ 
Vdxt'^^ = 63 - 2xt^^ - xf > (28) 

Axf^^ = 43 - xl*^^> - x^^^) ' 
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Having the ith approximation of all three ^'s, the (i+1) 
approximation of x^ is obtained from the first equation. 
Then, knowing x['''^^\ the second equation can be solved 
for xl,*+^^ and thereafter the third equation for x^-j'^^K 

Going back to the question of the convergence of the 
iteration equations (26), this equation can be written in 
matrix form as 
/25 
( 10 
V 4 
This in turn may be written 

Bx^'^^^ = b - Hx^^\ (30) 

where E and H are the square matrices shown. Note that 
this latter equation could be obtained directly from equa- 
tions (21), which can be written in the matrix form 
A X = b,hy observing that A = E -\- H. 

If both sides of (30) are multiplied by the inverse of E, 

/0.04\ 
£-i = (0.10) 
\0.25/ , 
we obtain the equation 

_^(w) = B-^h - £-1 H ^<« . (31) 

Now, since E~^ h = x'-°'>, the solution of the diagonal equa- 



tions, E x'-°'> = h, given in (22), and letting F 
equation (31) can be written 



.(i+i) 



= x^ 



(i) 



■\- P X 

Written out, this is just equation (27). 
In particular 



E-^H, 
(32) 
(33) 



X 



(2) 



+ F x^^^ = (1 H- F H-i?2) ;ir(«) 



.(») 



= ;t;(0) +i?^(»-i> = (1 -\-F -\-F^ -\- ... + i^")^^°^ 



thus, the convergence of the process reduces to the question 
of the convergence of the series 



£ 



pi _^.(0) 



(34) 



The latter can be shown to require that the characteristic 
values of F, in absolute value, be all less than unity. These 
characteristic values are the possible values of the constant 
A in the equation 

Fxp = \xp , (35) 

where 

is any vector chosen so as to satisfy (35). Such a vector 
is called a characteristic vector. 

Generally there will be three i/^'s and three correspond- 
ing A's, that satisfy this equation. 

Thus, in place of (35) we may write 

F ij/i = Xi x{/i , i = 1,2,3. (36) 

For n equations the number, of course, will be n instead 
of 3. 



I shall not prove the above requirement for convergence, 
except in the following plausible way. Let us assume that 
the characteristic vectors i/'i, xf/^, \pz ^^^ three linearly inde- 
pendent vectors, so that x^°^ can be expanded in terms of 
them, 

^(0) =k,xl;, + k,^,-\-k^xf;, , (37) 

where the ^'s are suitable constants. Then 

F^w = k,F^,-\-k,Fil,, + k,Fip, , 
or by (36) 

F x^°^ = ki Ai xl/j_ -\- k^ A2 V2 + ^3 ^3 i's • 
Repeated application of F to x^°\ therefore, gives 

F' x^'^ = k, \{ Vi + K Aij i/'2 + h K n , 
and hence, if 

1 Ai I < 1 , for I = 1, 2, and 3, 
lim F*.r(°) = 0. 
«'— » 00 
Moreover, if | A^ | > ] Aj | and | A^ ] > | A3 | and n is 
sufficiently large F»^^ x^""^ = k,\T^ xp^ = A^ F» jr<°) . 
Thus, the series (34), as far as convergence is concerned, 
acts like a geometric series with a ratio given by k^. Since 
I Ai |< 1, this series should converge. 

Finding Characteristic Values of Matrices 

The above discussion of the iteration methods points up 
the need to find the characteristic values A{ for a matrix. 
This fundamental problem is very interesting. Let me make 
several observations concerning it. 

Let F again be the matrix under consideration, but let it 
be used to represent any matrix being studied. We require 
the characteristic values Ai of equations (35) and (36). For 
this purpose, let us introduce the unit matrix, 

/I 0\ 
7=0 1 (38) 

\0 1/ , 

then (35) can be written 

{F -kl)xp = 0. (39) 

In expanded form, (39) is written as follows 

F,, F,,-X F,, \ lc,\ = . (40) 

\^31 ^32 PzZ-^l \czl 

Equation (40) represents three homogeneous linear equa- 
tions for the components Ci, c^, and Cg of xp. 

A solution of these equations, other than trivial solution 
^^ = ^2 = ^3 = 0, is possible only if the determinant of the 
coefficients is zero, that is 

-t'li A x^j2 i^ \z 

D (A) = i?21 ^22-A i^23 = . (41) 

■''31 -'^32 ■'^33 ^ 

Equation (41) leads to a polynomial equation of the «th 
degree in A (here w = 3) ; therefore, there are at most n 
characteristic values. 

One of the methods for finding the largest of the charac- 
teristic values, say A^, is indicated by our previous discus- 
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sion. If ^ is any initial vector, which can be expanded in 
terms of the characteristic vectors, as ;r^°^ was in {27), 
then repeated appHcation of P leads to a vector that is 
nearly xp^ times a constant. When one reaches this point, any 
further multiplication by P multiplies this vector by Ai, 
essentially. There are several methods based on this fact. 

Another method, one with which you may not be familiar, 
is to solve for the roots of D (A) directly by the use of the 
method of false position ; that is, we substitute some value 
of A in the determinant and evaluate the determinant nu- 
merically. We repeat this for several neighboring A's. This 
gives us a few points on the plot of D (A) versus A. If D (A) 
has opposite signs for two values Ao and A?,, then, since 
D{X) is a polynomial and hence continuous in A, it must be 
zero somewhere between these values. The method of false 
position estimates this value by assuming a linear variation 
of D{X) between Aa and As, and is conveniently coded for 
automatic computation. 

Since each evaluation of i^ ( A) requires evaluating a deter- 
minant of the Mth order, and this requires (l/3)«^ multipli- 
cations, the process is open to serious objections. If it is at 
all feasible, it is desirable to evaluate the coefficients of 
powers of A in D{\), since once this is done, the task of 
obtaining D{k) for some value of A is reduced to n multi- 
plications. The great advantage of methods of this sort is 
that all of the characteristic values can be evaluated at least 
in principle, and not just the largest. 



I shall close by pointing out one of the simple ways in 
which one may obtain an upper bound for the absolute 
value of the largest characteristic value A^. This can be ac- 
complished by considering equation (40), which must be 
satisfied. For the moment, assume that | Ci | is the largest 
of the three numbers | c^ | , ] Cg | > and \ c^\, the absolute val- 
ues of the components of the characteristic vector xj/ corre- 
sponding to Ai. Then from the first equation arising from 
(40), we have 

AyC^ — -^11 '^l "T" -^12 ^2 ~r -''13 ^3 

or 

This is just the sum of the absolute values of elements of 
the first row of P. 

If 1^2! is the largest of the constants, it can be shown, 
from the second equation, that |Ai| is less than the sum of 
the absolute values of the elements in the second row. Like- 
wise for kg] largest, the absolute values of the elements in 
the third row are summed. Without making any assump- 
tions as to the relative sizes of the \ci\, nevertheless, the 
following rule can be stated : If the absolute values of the 
elements of each individual row are summed and the largest 
sum chosen, this sum must exceed the largest characteristic 
value. This upper limit is very helpful. 

There are, of course, still other schemes for determining 
an upper bound on the size of the characteristic values. 
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AN ALTERNANT MATRIX is of the form as 
shown in Figure 1. It is a square matrix of order « in 
which the elements of each row are increasing powers from 
to N —\ of A^. Thus the elements of the first column 
are one. 

According to Aitken/ the reciprocal of such a matrix can 
be written. The elements of columns 1, 2 and n are shown 
in Figures 2, 3, 4, respectively. 







A, 


At 


A^-' 








A, 


A?, 


A^-' 








A, 


A! 


At^ 








A, 


Al 


^S-X 








Ay 


Al '. 


JN-l 

















Figure; 1 



1st Coi<umn 
A, • A,... As 



{A,- A,) {A,- A,) {A,- A,)... {A,- Ay) 



A^A^ + A^A^ + ■ • • + A^Ay + A^A^ + ... -\-A^An + • • • + ^y-i^y 
+ (A,-A,) {A,-A,) (A.-A,) . . . (A,-Ay) 

A, + A, + A^ + A, + ... + Ay 



+ 



{A,-A,) (A,-A,) {A,-A,) . . . (A^-Ay) 

1 

(A,-A,) (A.-A,) (A.-A,) . . . {A,-Ay) 



Figure: 2 



2nd CoLUMl-f 

Aj • A^ • Aj • A^...Ay 

{A,- A,) {A,- A,) (A,- A,)... (A,- Ay) 



+ 



A^A^ + A,A^+ ...+A^Ay + A^A^+ ...+A^Ax+ .■..+Ay.,Ay 
(A.-A,) (A,-A,) (A,- A,). ...{A,- Ay) 
A, + A, + A, + A, + ... + Ay 



+ 



(A.-A,) {A,- A,) {A,- A,)... {A,- Ay) 

1 

{A,- A,) {A,- A,) {A,- A,)... {A,- Ay) 



Figure; 3 



± 

A,A,A, 




Nth C01.UMN 
A-^ ■ A^ • A^ • A^ . . . Ay^i 


+ A,A,Ay., + . . 


-\- Ay.^Ay.^Ay., 


(Ay-A,) {Ay-A,) {Ay -A,) ... {Ay -Ay. 

+ A:,A^A, + ... +A,A,Ay_, + A,A,A, + ... 


A,A, 


+ 


A,A, + . 


{Ay-A,) {Ay-A,) {Ay-A,) . 

. . 4- A.,Ay^ + A.,A, + . . . + A^At, 


.. {Ay- 
r-x+.. 


-Ay.,) 
4" Ay., Ay 


^ 




A 


{Ay- 
1+^2 + 


-A,) {Ay-A,) {Ay-A,) . . . {Ay- 
A, + A,+ ...+Ay., 


-Ay.,) 




{Ay- 


-A 


) {Ay-A 


,) {Ay- A,)... {Ay- Ay.,) 

1 


' {Ay- 


-A 


) {Ay-A 


,) {Ay- A,)... {Ay- Ay.,) 



Figure; 4 
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Note that all of the denominators in any one column are 
identical. The numerators are formed by taking various 
combinations of the values of Ajf, Ajj.^, A^.^ , • • . , Aq. The 
forms for denominators and numerators are shown : 



Denominator 



TT (^i - ^r) 



r=l 

i = column index 



TT = product of indicated quantities 
Numerator JiiiL 

r=l 

Terms taken A'' — 1, A^ — 2 , . . . , 3, 2, 1, at a time for 
rows 1,2,2) ,..., N, respectively. 

It might be of interest to consider the number of terms 
in any numerator. This can be done by finding the number 
of combinations of (A'^— l) things taken (N — j) at a time. 
For example, in a matrix of order 13, the numerator of the 
sixth row would contain 792 terms : 

N — order of matrix 
y = row index 



The algebraic signs of the terms in the even-numbered 
rows, counting from the bottom up, are negative. 

If /\A is constant and positive, where /\A is the differ- 
ence between successive terms of column 2 of the alternant 
matrix, the denominators can be simplified : 

AA = {As - A J.,) = 8 

D^ = (-1)^^* (A^-1)! 0! 8^-1 

D^ = i-iy-' (N-2)\ 1! 8^-^ 

D^ = i-iy-"' (N-3)\ 2! 8^-^ 



Djr.^ = (-1)^+* 1! (N-2)l 8^-1 

Dn= i-iy^' 0! (A^-1)! 8^-^ 

D^, D^, D^ , . . . , Dn are denominators of columns 1, 2, 3, 
. . . , N, respectively. 

To facilitate the use of IBM punched card machines, the 
solution was written in a different form. This eliminates 
finding as many products and combinations of products as 
in previous forms. The first, second, and A^'th columns for 
the inverse matrix are shown in Figures 5, 6, and 7. 



P — A^ ' A^ • A^ • A^ . . . Ajf 



1st CoIvUm n 



Row 1 



P 



A,D, 



Row2:-^r^+-i-+^+ . . . +1+-11 

^i^iL^^v-1 -^jv An_2\^n Aj^^^J Aif_^\AN Ajf-i A]sf-2/ 

A2\An ^AT-i An_2 ^3/ J 

Row 4--^—i— ^ — +-^ r 1 , _L_L._]_(JLj LAI 

A^D-i {A]ff_2 An--^ A}f A]ff^^\_Ajf_^ An Ajf^2\^N ^a'^-i/J 

..,.. 1 r 1 . j__^ j_ M +_j_\+_i_ / 1 I ^ + Ml 

An-^L.'^n-i An An-2\-^n An-iJ An-3\An A^-i An-z/J 

A2\_An-i An An-2\An An-iJ An-s\An An-^ An-^I 
Az\An An-1 An-2 At^/j J 

p , , p r 1 1 1 1 1 111 

Row A' : -r-pr- i—r'-r'—. • — ; — • —^ — • -r- = tt 

A,D,IA2 A, A, An-2 An-^ An} D, 



+ 

+ •••+- 



FiGURie 5 



Row 1 



2nd CoIvUmn 
P 



A,D, 



Row2:-^^r4-+4-+-7^+ •••+4-1 

Az \A^ An A^JJ 

Row 4 -.-^(^ . ^ . i+^ [-- • -+-^ ('-+-)1 

AzDzlA^-Ti An A-y An-2\_An A.^ An-i\Ai AnJ_\ 

An-z\_An A^ An-.i\Ai An/ An-2\Ai An An-i/_J 

As\_An A^ AN-x\Ay An) An-2\Ai An An-iJ 
+ ■ ■ ■ '^A'XaI'^An'^aZ,'^ ' ' ' "^^ajj J 

^ ,, p r 1 1 1 1 1 "1 1 

^2^2 L^3 A, An-^ An A,J D^ 

Figure 6 



Row 1 :- 



Nth CoIvUmn 
P 



AnDn 
Row 2:-^ [-7^+-^+-^+ • • • +j-+^1 

Row 3 •~r~Fr~ ~i — • ~i )r~2 — ( 1 l~"l — ) 

An^n\_'An-2 An_^ An-s\An-i ■^n-2/ 

An-4: \An-i An-2 An-sJ 

Ai\An-i An-2 An-s Ag -^2/ J 

Ro„ 4 :^- 1^ . -!- . ^+^ f— • ~+-^ f— +-^)1 

AnDn \An-3 An-2 An-i AN-ii_AN-2 An-i An-3\An-i An-2/ J 

_^ 1 r_j 1-+-J- ("-1-+ J_V-J_ M I 1 I 1 \l 

An-5 \_An-2 An-i An-3 \An-i An-2/ An-^ \An-x An-2 An-3/_\ 

+ • • • +^ f— • —+—(—+—) 

Ai\_An-2 An-i An-3\An-i An-2/ 

+H:^+:?-+:r-V ■ ■ ■ +H-^+t~+t~+ ■ ■ ■ +z)] } 
Ro^ ^ p n . ± . j_ 1 1 ?_l=-i_ 

AnDn L^i ^2 A^ Ans An-2 An-^J Dn 

Figure; 7 
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P = A-^A^A^ . . . An D^, D2 , . . . , D^ are the denomi- 
nators of columns 1 to N, respectively. 

For actual calculation procedure the values of P, 
D„D„D,,...,Dn 
1/A„1/A„1/A,,...,1/An 

P/AJ)^, P/A^D^ ,... , P/A^Dn are calculated first. 
Note that the last values are the elements of the first row 
of the reciprocal matrix. 

N decks of cards are made up, each containing AT cards. 
The decks are made up from a circular arrangement of the 
values of \/An, 1/^jv-i , • • • , V^i- The last card in each 
deck is '^/Ai replaced by P/AiDi. These cards also contain 
the values offset gang punched on the following cards, as 
shown below. 

1st Coi^umn 
J_ 

1/%., 1/An 1/An-^(\/An) 

1/An-, l/AN-^ I/An-A^An -\- 1/An-^) 

1/An-^ \/An-^ 1/^^_3 {\/An + 1/^2^-1 + l/^i^-2) 



1/^2 1/^3 

PJAJ)^ 1/A, 

2nd C01.UMN 
1/A, 
I/An 
I/An-, 



P/A,D, (I/An + I/An-, + • • • + 1/^2) 



3rd Coi^umn 
l/A, 
l/A, 
I/An 
I/An-, 



Nth CoIvUmn 

^/An-2 
^/An-s 



The first group of values is used as multiplier while the 
second group is used as multiplicand after being progres- 
sively accumulated. The product for first column is shown 
below. The last card in the deck contains the element of the 
second row. 

For each successive row, the first group of values is re- 
produced, and the products are offset gang punched on 
following card. This process is done N — I times. Note 
that each time the process is completed, the number of cards 
in each deck with products other than decreases one, until 
the decks for the last row, the only card containing a 
product, will be the last card which is the P/AnDn card. 



I/An 
I/An-, 

\/An-2 

I/An-, 
I/An-, 



1st CoIvUmn 



\/lN ^N-i/ 

\An An-, An-2/ 






1/^3 

P/A,D, 



l/A, 
P/A,D, 



l/A, 
P/AnDn 



1. A. C. AiTKEN, Determinants and Matrices (London: Oliver, 
1945). 
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THE ONLY TYPE of calculation to be considered 
here is matrix multiplication. Time has not permitted any 
concrete work to be done on matrix inversion, but I shall 
have a few comments to offer on this later. In matrix work 
there are only simple arithmetric operations. This multipli- 
cation demands only two steps : 

Operation 1 : This is a shifting from electronic storage 
A (FS 1-2) into the electronic counter. For this problem, 
channel A is permanently connected to FS 1-2 (assigned) 
and channel B to FS 3-4 (assigned). 

Operation 2: This is an 8-digit by 8-digit multiplica- 
tion with the results in the electronic counter. 
The problem to be considered is the following matrix 
multiplication : 



AB 







..\= c 



The general term of the product matrix takes on the follow- 
ing form :^ 

h 

that is, each term is the scalar product of a row and column 
vector. There are two practical methods of calculation. One 
is to perform all the multiplications involving one row of 
the left-hand matrix ; this generates an entire row of the 
product matrix. The other is to perform all the multiplica- 
tions involving one column of the right-hand matrix, thus 
obtaining an entire column of the product matrix. The lat- 
ter has been chosen for reasons which will become obvious. 

Calculations 

The machine is loaded with the elements of one column 
of the right-hand matrix, the instructions for which will be 
found in Table I. The zero instructions for channels A and 
B indicate card reading. In Operation 1, as mentioned 
above, the quantity in FS 1-2 is shifted to the counter. The 
instruction for channel C is the code number for the storage 



register or accounting machine counter in which the indi- 
cated item will be held. The problem considered here is 
simple enough so that no shift is required. The elements of 
rows 1-5 are stored in the accounting machine counters, 
using an adding code of 7. 

The entire bank 1 and all but one register in bank 2 of 
the mechanical storage (941) has been used, in addition to 
5 accounting machine counters. Of the two remaining 
counters, number 1 is too small and number 2 will be used 
to accumulate totals. Thus, the method to be used here can 
be used for multiplication when the right matrix contains 
21 or fewer rows. For larger matrices, elements containing 
fewer digits might be used with the registers split. 

Table II shows the general layout for the deck of cards 
to be used, and a detailed description of the clearing cards. 

Table III contains the description of one row of the left- 
hand matrix A. Each element of this matrix is card read 
over channel A. The corresponding element from the right- 
hand matrix B is called from storage on channel B. The 
operation 2 is multiplication, with the 72 in channel C add- 
ing the product in counter group 2. 

It should be pointed out that, with the exception of the 
clearing cards, only one row and one column have to be 
programmed. All other rows or columns, as the case may 
be, take on the same form. 

The clearing cards can be eliminated by changing the 
channel B coding of the last row of the A matrix to counter 
read out and reset (8), instead of just read out (7). The 
deck to be run through consists of column 1 of matrix B 
and all of matrix A ; column 1 of the product matrix C will 
be obtained. The row number of matrix A is used as a con- 
trol field to clear counter 2. The products that make up Cn 
will be listed ; however, all other elements will be tabulated. 

ThS standard methods of matrix inversion — any of the 
elimination methods to a triangle or to a diagonal directly 
— involve approximately eight to sixteen thousand inter- 
mediate results, depending on whether you carry the unit 
matrix in your calculation. 



47 



COMPUTATION 



Let's take for granted that we do not carry it along. We 
will just carry along the 400 elements of the 20 X 20 ma- 
trix in the inversion. Even this involves 8,000 summary 
punchings, which take in the neighborhood of 1.5 seconds 
each as compared to a 0.4 second card cycle. The obvious 
way is to avoid taking intermediate results out as much as 
possible. 

One way of accomplishing this is through the use of the 
enlargement method. To review this : one starts with the 
inverse of the upper left-hand element, which can be en- 
larged to the inverse of the upper 2x2 matrix through 
simple multiplications, additions, and subtractions. It is a 
function of the inverse of the single element, the additional 
column, row, and diagonal element as shown below : 

^ai ^22 / "^ •^^"''' "'" ^'" ^'^'^ ' 
This enlargement then proceeds, increasing the matrix by 
1 row and 1 column in each step. Another alternative is the 
use of second order enlargement, which increases the order 
by 2 in each step. This general method appears to have 
several advantages ; summary punching and machine card 
passes are reduced considerably, and the problem of sig- 
nificant digits can be avoided by checking the intermediate 
inverses and iterating them for greater accuracy if nec- 
essary. 
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Tabi,^ I 

Right-Hand Matrix Coi^umn Codb 
Column A Oper. B C 



A -Entry 



1 y 00 1 00 

2 

3 

4 

5 

6 

7 

8 

9 
10 
11 
12 
13 
14 
15 
16 
17 
18 
19 
20 

All blank spaces indicate the same entry as that used in first line 



7Z 


Ki 


74 


Ks 


75 


Ki 


76 


Ki 


77 


Ki 


11 


^6. i 


12 


h^.i 


13 


^8. i 


14 


Kj 


15 


Ko.j 


16 


hi,j 


17 


^12. 3 


18 


Ks.j 


21 


^14. i 


22 


^15,/ 


23 


^16. i 


24 


i>n,j 


25 


hs.j 


26 


^19, i 


27 


^20, } 


t used 


in first Hi 



Card No. 



Tabi,^ II 
CivE;aring Cards 

A Oper. 



C* 



81 
82 
83 
84 
85 
86 
87 



00 



*This first card channel C clears the 604 electronic counter. 



Gene^rai, Ddck 

(Obtains jth column of product matrix) 

1. Seven clearing cards 

2. yth column of B (20 Cards) 

3. Matrix A in row order (400 Cards) 

4.* 2 blank cards 

*These are necessary to print the list result when an intermediate 
control break is used. 



Tabids III 

Lei^t-Hand Matrix Row Code; 
Rom Column A Oper. B C A-Entry 



00 



73 



72 



i 1 

2 

3 

4 

5 

6 

7 

8 

9 
10 
11 
12 
13 
14 
15 
16 
17 
18 
19 
20 

All blank spaces indicate the same entry as that used in first line. 

re^s'eirdnce; 

1. Kaiser S. Kunz, "Matrix Methods," pages 37-42. 
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fl^i, 6 


12 
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13 
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14 
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15 
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16 


«t, 11 


17 


«i 12 


18 


^i, 13 
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«i, 14 


22 


«i, 15 


23 


«{. 16 


24 
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<^i. 19 
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JS/lachine Nlethods for Finding Characteristic 

Roots of a Matrix"^ 

FRANZ L. ALT 

Computation Laboratory , National Bureau of Standards 



THE PURPOSE of this paper is to describe a few 
expedients which can be appHed to computation of charac- 
teristic roots of matrices by means of punched card ma- 
chines. In the course of two problems of this kind, recently 
handled by the Computation Laboratory of the National 
Bureau of Standards, some of these methods or variants of 
methods were actually tried out on cards, and some others 
were considered and laid out without actually being carried 
through. In both cases the general type of method used was 
suggested by the originator of the problem. 

The; first of these examples was of the conventional type : 
given a matrix A of order w (in the example, n = 14), with 
elements aik representing the approximate distribution of 
elastic forces in an idealized airplane wing, to find the three 
characteristic roots with greatest moduli. For finding the 
dominant root (i.e., the one with greatest modulus) there 
is the common method of starting with a trial vector 
3;«» = {y[°\ y(2°^ , . . , y\^^), and multiplying it repeatedly 
by the given matrix. Thus, y^^^ = Ay^^~^'>. This method is 
described, e.g., by Frazer, Duncan and Collar.^ It is an 
excellent method for punched card machines, since the 
multiplication of a vector by a matrix can be carried out 
very simply. One minor trouble that arises is that after 
repeated multiplication the numbers fall outside the range 
of decimal digits which had been allotted to them on the 
machine. This is prevented by "norming" the vector after 
each multiplication by the matrix. We accomplished the 
norming by making the last component of the vector equal 
to 1 after each step. Thus (with a slight change in nota- 
tion) we set 



y(k) 



y 






This method would fail in case the matrix were such that 
the last component of the characteristic vector happens to 
be very small compared to the other components. Obvi- 
ously, other norming methods could be used which avoid 
this failure. However, it seems preferable to have a simple 

*The preparation of this report was sponsored by the Office of Air 
Research, USAF. 



method, which fails once in a hundred cases but saves work 
in the remaining 99 cases. With this norming convention, 
the factors 3;^^' converge to the dominant characteristic 
root, and the vectors 3;^''^^ to a corresponding characteristic 
vector. 

The computations were performed on the 602-A calcu- 
lator. The 602 or 604 would have been equally suitable, 
since there is no great amount of number storage, A ma- 
chine with two card feeds, such as the Aberdeen IBM 
Relay Calculators, would have been superior, because in 
this case it would have been possible to feed the matrix 
cards into one card feed and the vector cards into another. 
Since we had no such machine available in our laboratory, 
we proceeded as follows : 

The matrix elements are punched into a deck of cards, 
one element to a card. This deck is reproduced as many 
times as we expect to have iterations. Before starting any 
one iteration, one of these decks is prefaced by a small deck 
containing the latest approximation to the characteristic 
vector (in the case of the first deck, the chosen trial vector 
y'-^^), the combined deck is sorted by columns, the vector 
elements are gang punched into the matrix cards, then the 
deck is followed by a set of summary cards, sorted by rows, 
and put through the 602-A for performing the matrix multi- 
plication. This operation produces the unnormed, new ap- 
proximation to the characteristic vector, punched into the 
summary cards. These are then sorted out and put through 
the 602-A again for the norming process. 

To obtain the second characteristic root, the method of 
"sweeping-out" the first characteristic vector is used. That 
is to say, proceed exactly as for the first root, but after each 
iteration subtract from the iterative vector y^^^ a certain 
multiple of the first characteristic vector. The same process 
can be carried out for subsequent characteristic roots. In 
these cases it is desirable to punch each component of each 
iterative vector in several successive summary cards, one 
for each of the previous characteristic vectors to be swept 
out. 

In the actual example carried out in our case, there were 
additional computing steps brought about as a result of the 



49 



50 



COMPUTATION 



fact that the equations of motion of the airplane wing were 
referred to a moving coordinate system. This requires an 
adjustment after each iteration ; the computation is similar 
to the sweeping-out of earlier characteristic vectors. 

The vectors converge reasonably well, except in cases 
where there are two characteristic roots with equal or al- 
most equal moduli. We did not run into any such cases. 
Nevertheless, we felt it useful to speed up the convergence 
of the process. A method which we used for this purpose is 
the one described by A. C. Aitken^ and called by him "the 
delta-square process." It consists in taking three successive 
approximations to the desired characteristic root, say, Vt-i, 
z't, and Vt+i, and extrapolating from them to the desired 
root by using the expression 

Vu-, Vti — v-t 
vu^ - 2vt -f Vt-i 
The same method can be applied to find directly a close 
approximation to the characteristic vector. 

Another method, which we discussed but have not yet 
used, consists in subtracting from all terms of the principal 
diagonal of the matrix a suitable constant, chosen in such 
a way as to increase the ratio between the dominant and 
subdominant characteristic root. (The subdominant root 
is the one with second-largest modulus.) Suppose, for ex- 
ample, that it is known that all roots are real (as, for in- 
stance, in the case of symmetric matrices with real coeffi- 
cients), and the largest root is estimated to be 10, the sec- 
ond largest 9, and the smallest 1. By subtracting 5 from all 
elements of the principal diagonal of the matrix, a matrix 
is obtained whose characteristic roots are smaller by 5 than 
those of the original matrix ; that is, the largest root has 
become about 5, the second largest 4 and the smallest —4. 
The ratio of largest to second-largest, in absolute value, is 
now 5 : 4, whereas previously it was 10 : 9. Since the speed 
of convergence of the iteration process tends to increase 
with the size of this ratio, the process is likely to converge 
faster for the modified matrix. In general, the constant to 
be subtracted, in case all roots are real, is the arithmetic 
mean between the root nearest the dominant root and the 
one farthest away from it. It is necessary, of course, to have 
estimates of these roots in order to apply this method. 
One; ve;ry ostein encounters matrices which might be called 
"almost triangular." The name "triangular" shall be ap- 
plied to matrices in which all elements above the principal 
diagonal are zero. By "almost triangular" is meant a matrix 
which has only a few nonzero elements above the principal 
diagonal, and those are all bunched close to the diagonal. 
To be exact, an wth order matrix A with elements aik will be 
called "almost triangular of degree t" if aik = for k — i > ^, 
where t is some integer between and m — 1. There is no 
restriction on the elements below the principal diagonal. 
However, some of the statements which will be made 
toward the end of this paper apply only to "almost diagonal" 
matrices, which are defined analogously by Uik = for 



\k — i\ > t; that is to say, both above and below the prin- 
cipal diagonal all elements except those close to the di- 
agonal are zero. 

For a completely triangular matrix, that is, t = 0, no 
computation is required. The characteristic roots are equal 
to the elements of the principal diagonal. 

Now take the case t = 1. Consider the matrix A — A/ as 
the matrix of a system of simultaneous homogeneous 
equations. 

(an — \) x^+ Oi2 ^2 =0 

«21 ^1 + (<^22 - A) ^2 + «23 ^i =0 

^»l •^i + • • • + i^nn — X) Xn = 

Our problem is to find those values of A for which the sys- 
tem has a solution. Because of the "almost triangular" 
character of the matrix, the first equation contains only the 
first two unknowns, the second equation only the first three 
unknowns, generally the ^th equation, only the first k -\- \ 
unknowns. For simplicity of presentation, let us assume 
first that a^fc y^ for k — i = 1 . Let us choose a particular 
value of A and ask ourselves whether the system of linear 
homogeneous equations has a non-trivial solution for this 
particular A, that is, a solution in which not all of the un- 
knowns are equal to 0. It can easily be seen that because of 
our assumption that ai^ i+i y^ 0, the value of the first un- 
known in any non-trivial solution is not zero. And since 
the system is homogeneous, an arbitrary nonzero value can 
be assigned to x-^^, for example x-^ = I. Now substitute 
jfj = 1 in the first equation and obtain X2, then substitute 
Xi and x^ in the second equation and obtain x^, etc., down 
to the (w — l)st equation from which the value of Xn is ob- 
tained. If, now, all these values a'i, X2, . . . , Xn are substi- 
tuted into the wth equation, this equation may or may not 
be satisfied. The result of the substitution in the left-hand 
side of the equation is, of course, a function of the particular 
value of A chosen initially, and it may be designated by 
£(A). If, and only if, -E(A) = 0, A is one of the charac- 
teristic roots of the matrix. Now, the value of ^(A) for a 
number of different A's may be computed, until enough 
values of the function H(A) are obtained to determine its 
zeros, either graphically or by inverse interpolation or some 
other method. 

This method of obtaining characteristic roots was de- 
scribed by Myklestad^'* and Prohl.^ They described, inde- 
pendently of each other, the application to two different 
engineering problems, but apparently without noticing its 
general applicability and import. To Dr. A. Gleyzal goes the 
credit for having noticed this and for having generalized 
the method to cases of f > 1. 

If these substitutions are carried out for a number of 
different values of A, let us see how the values of the un- 
knowns .^1, X2, etc., depend on A. Of course, x^ is chosen 
arbitrarily, and it does not matter how it is chosen, as long 
as the same x-^ is substituted with each value of A. For the 
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sake of definiteness, let ;i'i = 1 . It is seen at once that X2 is 
a linear function of A; specifically, x^ = — (flu — A)/ai2' 
Similarly, it is seen, by substitution, that x^ is a quadratic 
function of A, and generally that Xjc is a polynomial in A of 
degree k — 1. This property is of very great assistance in 
checking the computations. It is also evident that £(A) is a 
polynomial of degree n and, as such, has n zeros. (These, 
of course, coincide with the n characteristic roots of our 
matrix.) Theoretically, therefore, it is sufficient to compute 
B(X) for n -\- 1 values of A. Thereafter the polynomial can 
be extrapolated indefinitely by differencing. In fact, how- 
ever, one might lose too many significant digits in this 
process. It is, therefore, preferable to compute B{\) for 
more than n values of A, say, something like 2m values, and 
to distribute these as far as possible equidistantly over an 
interval which spans all expected values of A. I might also 
mention that, in the example with which we were con- 
cerned, the question was not to find all characteristic values, 
nor to find the largest or smallest in absolute value, but to 
find those characteristic values which are located in a given 
interval. For this type of problem this method is particu- 
larly well suited. 

Sometimes one finds a matrix which is not almost- 
triangular as it stands, but which can be brought into 
almost-triangular form by suitable rearrangement of rows 
and columns. (It is necessary to apply the same permuta- 
tion to both rows and columns in order to preserve the 
characteristic roots.) In the case of any matrix with many 
zeros, it is worth looking for this possibility. 

Let us, now, drop the requirement that all «{, j+i be differ- 
ent from zero and assume that one of these elements, say 
o^-, y+i is equal to 0. In this case we can obtain the charac- 
teristic roots of the matrix A by partitioning it into a 
matrix ^^ of order /, consisting of the elements in the upper 
left-hand corner of A, and a matrix A^ of order n — ;', con- 
sisting of the elements in the lower right-hand corner of A. 
Each root of ^4 is a root of either ^^ or A 2, and vice versa. 
The roots of ^^ and A^ can be found by the method de- 
scribed above. If there are several zeros among the elements 
just above the principal diagonal, the matrix A is parti- 
tioned into a correspondingly larger number of sub- 
matrices. 

We now turn to the case f > 1. The essential features of 
the method can be fully explained for t = 2. Generalization 
for greater values of t will be self-evident. The basic idea is 
again to attempt to solve the homogeneous linear system of 
equations for a number of particular values of A. 

The system now has the form 

(«11 — A) JlTi -f- ^12 ^2 + «13 •*"3 =0 

«21 ^1 + («22 — A) .«-2 + «23 ^3 + «24 ^4=0 



(1) 



^n-2, 1 -^1 "I 

^«-i, 1 •^i "r 
^»l ^1 "T" 



+ an- 



= 
= 

+ {dnn — X) Xn = 



As before, consider first the case in which all Aj, {+2 are dif- 
ferent from zero. For any given A, in the case ^ = 1 it was 
necessary to start with an assumed value of x^. For t =■ 2, 
it is necessary to start with assumed values for both x^ and 
x.^. For example, start with x^ — \, x^ = and obtain suc- 
cessively from the linear equations the values oi x^, x^ , . . . , 
Xn. By this time the first n — 2 equations have been used. 
Now all the x's can be substituted into the last two equa- 
tions. This gives a value ^i(A) for the left-hand side of the 
(n — l)st equation and a value /i(A) for the left-hand side 
of the «th equation. 

We now repeat the process with two different initial 
values for x^ and x^, say, x^ — 0, x^ = 1. For the left-hand 
side of the two last equations two new values e^ik) and 
/2(A) are obtained. In general, all four values e^, 62, /i, and 
J2 will be different from zero. To find the characteristic 
values of the matrix, notice the following: any non-trivial 
solution of a system of equations will start with certain two 
values x^ = a^, X2 = 0-2. (If all fli, i+2 =7^ 0, then x-^^ and x^ 
do not vanish simultaneously.) The values of the remaining 
unknowns in such a solution are expressible as linear com- 
binations of the unknowns, obtained in the two basic solu- 
tions before, with weights a^ and a^. Substitution of these 
unknowns in the last two equations gives two values 
^(A) = «! ^i(A) + a2 e^iX), /(A) = a^ j^{k) -|- ag /2(A). 
For a non-trivial solution, ^(A) and /(A) must vanish 
simultaneously. For this to be possible it is necessary (and 
sufficient) that the determinant 

^i(A) 



Z)(A) = 



M 



hW hW 



be equal to 0. This determinant is a function of A whose 
zeros coincide with the characteristic roots of the matrix. 
Just as before, we select a number of values of A and evalu- 
ate the determinant D(k) for each of them and then find 
the roots of jD(A), either graphically or by interpolation. 
To summarize, the steps taken are as follows : 



Assume 




x^^'^ 


= l,;i;/2> 


= 0, ^1 = fli 


Assume 




X2^^^ 


= 0, ^2^'^ 


= 1, ;ir2 = ^2 


Solve eq. 


lof(l) 


-*3 


x,^-^ 


X^ = Cl-iX^ 

+ «2^3^'^ 


Solve eq. 


2of(l) 


X^^^^ 


x,'^^ 


x^ = a^x/^^ 

+ ^2^4^^^ 



Solve eq. M — 2 
of(l) 



(1) 



(2) 



(1) 



. Xti 



Xn — fli-^« 

+ 0'2Xn''^^ 

Substitute in 

eq. M — 1 of (1) ^1 ^2 ^ = ^1^1 + ^^2^2 

Substitute in 

eq. w of (1) /i /a / = aji -h O2/2 

The terms in the last column above need not be com- 
puted ; they are listed here only to illustrate the explanation. 
The values oi x^S^^ and Xu^'^'> , considered as functions of A, 
are polynomials, so that their computation can be checked 
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by differencing. Likewise the determinant D{X) is a poly- 
nomial. It is possible to prove directly that D{\) is of de- 
gree n in A. This proof is not given here, since it is not 
needed for the argument. 

Cases in which one or more of the coefficients a^, i+2 van- 
ish need to be treated specially. It seems most economical 
not to complicate the machine method of computation by 
allowing for these degenerate cases, but rather to treat these 
cases separately when they arise. 

In the general case of any value of t, the determinant 
D(X) will be of order t. Considered as a function of A, it is 
always a polynomial of degree n, regardless of its order. 

Since the evaluation of the determinants of higher order 
is laborious, the method given here is recommended pri- 
marily for the cases of ^ = 1, 2, or 3. These are just the 
cases which are most likely to occur in practice. 

The performance of computations on punched card ma- 
chines is straightforward. The coefficients of the matrix are 
punched into cards, one coefficient to a card. From this 
deck, a series of decks is prepared, so that t decks are avail- 
able for each value of A to be used. All of these are identical 
with the original deck, except that the value of A has been 
subtracted from the numbers in the principal diagonal. The 
cards containing the coefficients a^, i+t are characterized, by 
a special punch, as summary cards. It is expedient, but not 
necessary, to divide each row by fli, i+t, so as to make the 
latter coefficient equal to unity. No cards are required for 
coefficients which are equal to zero. The computations pro- 
ceed in a number of identical steps, one step for each un- 
known Xj. We are going to describe one of these steps. 
Suppose that the value of Xj has just been computed, by 
using the cards of row / — t. There is one such Xj for each 
deck of matrix cards, i.e., for each value of A and each com- 
bination of assumed values x-^, . . . , Xf. Each Xj is punched 
into the corresponding summary card. 

Now sort all matrix cards on the column number, select- 
ing each yth column. Automatically, for each deck, the sum- 
mary card is in front and is followed by the remaining cards 
of the yth column. Feed the cards into the multiplier (either 
the 602, 602- A or the 604 could be used), use the value of 
Xj, as read from the summary card, as a group multiplier, 
and punch the products aijXj into the card corresponding to 
ttij. (Alternatively, it would have been possible to use the 
reproducer instead of the multiplier and to gang punch the 
values Xj themselves instead of the product.) 

Next, select the row ; — ^ -f- 1. In this row each card 
except the summary card has a product, aikXjc (or in the 
alternative procedure a value Xk), previously punched into 
it. This is so because the cards of the jth column have been 
punched in the preceding step, the cards of earlier columns 
have been punched in earlier steps, the cards in column 
y -|- 1 are the summary cards in this row, and cards for 
columns following y -f- 1 do not exist in this row, since all 
corresponding coefficients are 0. 



Now feed the cards into the machine and add all the 
products (in the alternative procedure, the products are 
formed in this step and added at the same time). When 
the machine reaches a summary card, it punches the sum 
of all products. This is the value of Xj+-^. Then select all 
these summary cards, place them in front of the matrix 
decks, discard all other cards of row j — t -\- 1, and from 
here on this sequence of operations is repeated. 

The polynomials ^(A) and /(A) are evaluated in the 
same way. The fact that Xj is a polynomial in A can be used 
conveniently for checking the computations by taking dif- 
ferences of sufficiently high order. Finally, ii t > 1, the 
determinants of order t have to be evaluated, either manu- 
ally or by machine, depending on how many there are. 
This in turn depends on the order of the matrix and the 
size of the interval being searched for characteristic roots. 

A considerable gain in efficiency over this method can 
be accomplished in the important special case of almost- 
diagonal matrices, for moderate size of t. In this case each 
row of the matrix contains at most 2t -\- I nonzero ele- 
ments. All sorting of cards is eliminated, the entire com- 
putation is performed in a single run through the 602-A 
multiplier. Where formerly the computation for a particu- 
lar value of y was carried out in succession for all A's be- 
fore going on to the next j, in this case all cards pertaining 
to one coefficient deck (i.e., to the same A and to the same 
choice oi X:^ , . . . , Xt) , are kept together, arranged by rows. 
At each step of the substitution, not more than 2^ -|- 1 
different unknowns Xi are needed. These are all stored iti 
the machine, the cards of a row are fed in, the coefficients 
ttijc read off the cards, and multiplied by the corresponding 
Xi, and the products accumulated and punched into the 
summary card of the row. Thereafter, the first of the 
stored ;r's is discarded, and each subsequent x is moved to 
the storage location of the preceding one. The last storage 
location is filled with the x which has just been computed. 
Now the machine is ready to receive the cards of the next 
row, and the whole process is carried out without ever 
stopping the machine. In our work so far, this method has 
been planned but not yet tried out on cards. 
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DISCUSSION 

Mr. Kimball: Dr. Alt mentioned that they had only one 
experience in the iterative multiplication. In 1945, using 
a complex node matrix of size 14 by 14, we obtained con- 
vergence to four digits in 33 steps of iteration, about half 
an hour for each step, using the 601 multiplier. 

Mr. Bell: Concerning practical problems that arise in 
evaluating the formulas that were derived: if you have a 
system of simultaneous equations and reduce the first to a 
triangular matrix and then by back substitution to a di- 
agonal matrix, you have essentially two procedures, and 
this complicates the machine work. 

We have found that there is a critical point beyond 
which you would consider the back solution and that the 
order of that matrix is quite high. I would say something 
like the 15th order, at least. The advent of the 604 has 
made the straightforward approach much simpler. In 
other words, instead of working on the first column and 
then eliminating it, leave it in the matrix. 

Another point is that if you divide and make your ele- 
ments 1 immediately, you are dividing by numbers whose 
size may be quite small, and that may make the size of 
the numbers go outside the limits of your field. 

We have found that a method which protects us in that 
respect is to leave the numbers as a number. Your equa- 
tion is then of a form where you subtract from each ele- 
ment a ratio multiplied by a number, and then the correc- 



tion tends to be small, if the dividing term is large, which 
will keep your numbers within size. 

We have done some work with the iterative methods 
without a great deal of success. We have found that the 
conditions of convergence are more difficult to determine 
than just going straight into the problem and trying to get 
a solution. 

In what we have done practically, in trying an iterative 
process, we have set it up and started it running. If we 
don't get solutions, if it begins to diverge, we stop, assum- 
ing that it is divergent. 

It seems to me that essentially those processes are de- 
signed where you do not have a machine that is capable 
of a grinding operation, such as the IBM machines. So 
that we almost always set the problems up for a direct 
solution. 

One other thing is that in problems of the form where 
you have a matrix that is symmetrical on both sides, and 
other special matrix forms where there are mathematical 
techniques that will give you much fewer operations, it 
means that you must have different procedures and dif- 
ferent methods for your operators, and that always slows 
you down. We have aimed to do as much of our matrix 
work by this one simple process as possible ; and, although 
the number of mathematical operations can be unneces- 
sarily large, the elapsed time is very much reduced, rather 
than trying to be elegant at every step. 

Chairman Hurd: A very good point. 



Solution of Simultaneous Linear Algebraic Equations 
Using the IBM Type 604 Electronic Calculating Punch 
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MANY METHODS exist for solving simultaneous 
equations with punched card accounting machines. The one 
presented here takes advantage of the speed and flexibility 
of the 604 electronic calculator. A 10th order matrix can 
be inverted in one hour by use of this method, which com- 
pares with approximately eight hours through use of relay 
multipliers. Furthermore, the method is extremely simple." 
The basic reduction cycle consists of : sort (650 cards per 
minute), reproduce (100 cards per minute), sort (650 
cards per minute), and calculate (100 car<is per minute). 
This cycle must be repeated a number of times equal to the 
number of equations. 

Theory 

Several variations of the basic elimination method can be 
used with the machine procedure outlined. The one de- 
scribed requires no back solution and is well suited to ma- 
chine methods. It is well known and will be described very 
briefly. 

The equations may be expressed in matrix notation as 
AX = C. C and X may have, of course, any number of 
columns. If A~'^ is desired, C becomes / and X becomes A~^ 
(see reference 1). 

The object of the calculation is to operate on the matrices 
A and C, considered as equations, so as to reduce ^^4 to a 
unit matrix, thus reducing C to X. 

Let M be the augmented matrix composed of A and C. 
Choose any row, k, of M and form M' such that 



micj = 






niij = niij — Wife — - ,ty^k . 

The ^th column of M' is zero, excepting the ^th row 
which is unity. Therefore, no cards are made for the Jfeth 
column of M\ 

Form M" from M' using the above equations, but a dif- 
ferent row for k. If this process is repeated until each row 
has been used and all the columns of A eliminated, the 
columns of C will have been reduced to X. 

*The value of the determinant of the matrix of coefficients can be 
obtained as a by-product of the process. See reference 1. 



For best accuracy and to insure that all numbers stay 
within bounds, the elements of M should be close to unity, 
and, if possible, the principal diagonal elements of A should 
be larger than the other elements. 

A column of check sums (negative sums of each row) 
appended to M provides an easy and complete check on the 
work. These check sums can be calculated by machine, but 
if they are manually calculated and written as a part of M 
they provide an excellent check on the key punching. Also, 
experience has shown that the agreement of the final check 
sums with X is an index to the accuracy of X. 

Machine: Procedure 
Layout 

The following card fields are necessary : 

A. Row (of M) 

B. Column (of M) 

C. Order (initially n and reduced by one each cycle 
until it has become zero) 

D. Common 12 punch 

E. Pivotal column 11 punch 

F. Pivotal row 12 punch 

G. Next pivotal row 12 punch 
H. Product or quotient 

I. Cross-foot or dividend 
J. Multiplicand or divisor 

Procedure, Using Rows in Order 

1. Start with M punched in fields (A), (B), (C) and 
amounts in (H). 

2. Sort to column. Emit 11 in (B) of column 1. 

3. Place column 1 in front and sort to row. Emit 12 in 
(G) of row 1. 

4. Reproduce cards. Emit 12 in (D) of all cards. Re- 
produce (A) to (A), (B) to (B), and (G) to (F). 
Reproduce (H) to (/) except that (H) oi the piv- 
otal column cards is gang punched in (/). Emit 11 
in (B) of the first card of each gang punched group 
(column 2 in this case). It is advisable to pass blanks 
on the punch side for the 11 in (B) masters. See 
note (3). 
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5. Sort the new cards to column [row 1 with 12 in (F) 
should automatically be the first card of each column] . 

6. Calculate on the 604. On the 12 in (F) masters, cal- 
culate (///) = Q, punch in (H) and store. On 
the following detail cards, calculate I-QJ and punch 
in (H). Gang punch 12 in (G) of row 2 from (D) 
by means of digit selectors. Gang punch (w— 1) in 
(C). See note (3). 

7. Sort to row. If check sums are carried, cards may be 
tabulated controlling on row. All rows should sum to 
zero except the pivotal row which should sum to — 1. 
Round-off errors will appear, of course. 

8. At this point, these facts exist : 

a. The cards are in order by row with column 2 first 
in each row [column 1 was not reproduced in 
step (4)]. 

b. Column 2 has an 11 in (£) which was emitted 
in (4). 

c. Row 2 has a 12 in (G) which was gang punched 
in (6). Therefore, the cards may be reproduced 
again as in (4), sorted as in (5), this time plac- 
ing row 2 in front, multiplied as in (6) gang 
punching (w — 2) in (C) and 12 in row 3, and 
sorted and checked as in (7). 

The process then consists of repeating this basic cycle : 
sort, reproduce, sort, calculate, until the order has been 
reduced to zero. Then all the columns of A will have dis- 
appeared, all the rows will sum to —1, and C will have 
become X. For a final check, multiply AX and compare 
with C. 

Note:s :^ 

1. The use of digit selectors in (6) can be obviated by 
placing the next pivotal row behind the pivotal row 
before (5) and gang punching the 12 from (F) to 
(G) in (6). It is felt that use of the digit selectors 
offers less chance for error. 

2. In (6), using a standard 40-program 604, the follow- 
ing limits on size of numbers seem to exist : 

Divisor : 10 digits 

Quotient : 10 digits 

Dividend : 12 digits 

Multiplicand : 10 digits 

Cross-foot : 1 1 digits 

The question then arises as to the best way to apportion 
these digits between decimals and whole numbers. Eight 
decimals is a good choice for many problems, and seven 

I'The writer will be glad to supply copies of the planning charts 
for the 604 and reproducer control panels used in this procedure. 
Address John Lowe, Douglas Aircraft Company, Inc., Engineer- 
ing Department, Santa Monica, California. 



would provide for all but the most disadvantageous cases. 
Since the 521 control panel is the same for any number of 
decimals, it may be advisable to have two or more calcu- 
lator control panels. 

3. In order to divide by ten digits, the ten-digit divisor 
is split into its first eight digits, x, and its last two 
digits, y. Then 

a(x -\- y)~^ = ax~^ — ax~^y -}- ax'^y^ — .... 
Only the first two terms of this series are calculated. 
If eight decimals are carried, 3;<10^, and eight- 
decimal accuracy is obtained if jr^.l . 

The following procedure provides eight-decimal accuracy 
when x<^l. 

a. As terms are calculated on the 604, they are checked, 
and if ^1. , an 11 is punched (not shown on schedule 
of fields). 

b. This 11 punch is gang punched to field (/) in the re- 
producing operation. 

c. In the next calculation, if this 11 punch is absent in 
the divisor field, the divisor and dividend fields are 
shifted two places to the left in reading. Thus, y be- 
comes zero, and eight-decimal accuracy is obtained 
at all times. 

After the first reproduction, or if pivotal rows are chosen 
manually (see note 5.), it is necessary to emit this 11 punch 
in the divisor field if the divisor is ^1. 

4. If several sets of equations are being handled simul- 
taneously, time can be saved by not sorting case in 
step (7) but making case the major sort in step (5). 

5. The nature of the equations may be such that rows 
and columns cannot be pivoted in order as outlined, 
but must be chosen so as to give the smallest quo- 
tients. In this event, the 11 in (B) and 12 in (G) 
must be emitted prior to the reproduction and their 
automatic insertion discarded at a sacrifice in speed. 

6. It is usually not economical to check every cycle on 
the accounting machine. Errors should be rare and 
will carry forward if they occur. One compromise is 
to let a given cycle check while the next one is being 
processed. 

1. WiniAM Edmund Mii^ne;, Numerical Calculus (Princeton Uni- 
versity Press, 1949), p. 26. 

2. Frank M. Verzuh, "The Solution of Simultaneous Linear 
Equations with the Aid of the 602 Calculating Punch," Mathe- 
matical Tables and Other Aids to Computation, IH, No. 27 
(July 1949), pp. 453-462. 
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DISCUSSION 

Mr. Turner: What do you do if B22 happens to be very 
small ? 

Mr. Lowe: In the manner I described, you would actu- 
ally pick the starting rows and column in sequence — that is, 
the first column in the first row and the second column in 
the second row, and so forth. It isn't necessary to do that. 
You can pick any one you want. In picking, pick the one 
that would give you the most advantageous numbers. In 
particular, we usually try to pick the one that gives the 
smallest quotients in doing this division. 

Mr. Wolanski: We have a method that is similar to this, 
but we always use the element that is greatest; we cannot 
say the first row or the first column. In the first column we 
use an element which is the largest ; when we do eliminate 
and get B21, and B^^ equals zero, we start in on our second 



column, and we pick the element that is the largest. 

Mr. Lowe: Our method for finding out if the numbers 
get too big is simply to punch out a few more numbers 
than we can use the next time and then sight-check the 
cards. 

Mr. Bell: In our handling of this problem we try to re- 
move judgment from the operation which the operator 
performs. We don't want him to have to look at it and 
evaluate and decide which term to use. So, in handling 
matrices — usually, in groups — we simply start up from the 
main diagonal. Perhaps ten per cent of the problems will 
go bad. We take that ten per cent and start down the main 
diagonal, and maybe ten per cent of those go bad. Well, 
then we have 1/lOOth left over, of the total working vol- 
ume, and those we actually evaluate and select proper big 
terms in order to make it behave. But by doing that the 
mass of the work is handled in a routine way. 
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THIS is a brief report on a study that is being made at 
RAND on the use of rational approximation in high-speed 
computing. The work we report upon was largely stimu- 
lated, in the first place, through appearance of the IBM 
Type 604 Calculating Punch, and our work was given 
further impetus by the reported development of the IBM 
Card-Programmed Electronic Calculator. 

The opportunity of doing away with the use of card 
tables thus presented itself to us, and we began to prepare 
for the day when compact approximate expressions would 
take their place in the art of digital computing. The subject 
of rational approximation then became a matter of increas- 
ing importance. We note in passing that proper use of the 
604 can eliminate the use of tables to a considerable extent. 
Thus, to give an example, one can evaluate a fifth (or even 
higher) degree polynomial in single-card computation on a 
604. The machine will then read an arbitrary value of x 
from a detail card and punch out P (jit) upon the same car4. 
This capability may be used to compute, for example, S-deci^^ 
mal sines over the quadrant from such a polynomial ex- 
pression. Simple rational expressions may also be computed 
in single card computation. Logarithms, square roots, and 
many special functions have been computed directly on the 
604 with considerable success. With the card-programmed 
electronic calculator, our needs for compact approximate 
rational expressions to univariate functions will be much 
increased, for in a sequence of calculations we shall often 
be required to "look up" a random value of a function in 
order to continue, and a bulky table is useless here. Some 
attention is also being given to the problem of multivariate 
approximation. Our study is largely of an empirical nature. 
We are compiling data on many instances of rational ap- 
proximation. A few comments on this data follow. In the 
remaining sections, a number of random topics pertinent to 
the study are discussed briefly. 

By the term "primitive approximation" we essentially 
mean the most accurate approximation that can be achieved 
in a given set of circumstances. We shall call the graph of 

\{x)=rix)-f(x), (1) 

the "error curve" of approximation. The reader will notice 
that the several error curves displayed in this paper have 
been truly leveled. That is, our approximations are "primi- 



tive." Our parameter values are then, of necessity, deter- 
mined to an excessive number of figures for practical 
purposes. These may be cut back to obtain "working" 
approximations. Each primitive approximation will be de- 
scribed by an accurate error curve, the primitive parameter 
values, location of roots n, location of extremal values Ci, 
and the common extremal values d. 

Thus, we approximate the common logarithmic function 
log;ir over (1/VlO, V^O) by the form 

iog*;r = c,$+c,e-^c,e + c, ^^ , 

and record the following data: 



C, = .8685,5434 
C3 = .2911,5068 
C, = .1536,1371 
C, = .2113,9497 



r^ = 1.000 
r^ = 1.446 
r, = 2.028 
^4 = 2.656 
r. = 3.098 



1.204 
1.722 
2.348 
2.920 
3.162 



d = .0000,0206 



VIO 



The example is an interesting and perhaps useful one. We 
notice that 

x-\- VIO 
\0g*X= .5-\-C,r, + C,r]^ + C,r,' + C,ri' 
is an equally good approximation to log.*- over the full in- 
terval (1, 10). 



.000002 - 



f(X) 



-.000002 - 




Figure- 1 
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Each function approximated is studied with respect to 
some reasonable sequence of forms, and not as an isolated 
case, lest one gain this impression from the example above. 
We will thus obtain empirical data concerning rates of con- 
vergence, location of roots, location of extremals, and con- 
vergence of parameter values. 

The Fitting of Rational Forms 

There are several important ways in which the problem 
of fitting a rational form may be linearized. One method is 
that of specification of roots. Another is that of specification 
of extremal deviations. 

We hope to make some useful comments upon the loca- 
tion of roots and extremals. Perhaps our final observations 
will merely be of an empirical nature. It has become appar- 
ent to us that, in comparable situations, one error curve will 
be much like another of the same order. Thus, consider the 
examples shown in Figures 2 and 3. (We shall actually rule 
out approximations of the type below as being non-com- 
putable. The next section will throw some light on this 
aside.) 



lV(x) = 



(1 + ^-) = 



V27r 



by 



by 



1 



4.03809 + .851-42^-^ + .13826.«-^ 



1 



2.56581 -f .7n5Sx^ -f .81788^^ 



While the functions approximated have quite different be- 
havior, the error curves are remarkably alike except for 
scale. 

Our first trial in making a rational fit is usually by speci- 
fication of roots. That is, we impose upon the form those 
values of x at which the error curve is to cross the .sr-axis. 




-.002 



CiX) 




Figure 2 



-.01 



Figure 3 

Then we compute the error curve on an evaluation sheet, 
and if we have a true feeling for the situation, all peaks will 
be of comparable magnitude. 

An evaluation sheet is simply a work sheet containing a 
list of key values of the function f(x) being approximated 
and a suitable computing setup for use in evaluation of cor- 
responding values of a given approximation /*(.*"). A final 
column allows for the evaluation oi €{x). 

To obtain the quality of approximation that we desire to 
exhibit, an adequate number of accurate function values 
must be incorporated in the evaluation sheet. Generally, we 
select about fifty or sixty values of the independent vari- 
able in the construction of a given evaluation sheet, and 
these are given to about three or more figures than the most 
accurate approximation we expect to obtain. With the pos- 
sible exception of a few extraneous values, our tabulated 
key values of a function f(x) will be for equally spaced 
intervals either in x or in some simply related transformed 
variable i = $(x). 

Once a crude approximation has been obtained and the 
corresponding error curve determined, we may employ the 
more refined method of specification of extremals to level 
the peaks as we have done in the several examples given 
above. Several computing cycles may be required to achieve 
the quality of approximation desired. 

In speaking of extremal deviations, we naturally refer to 
the error curve of approximation. Generally an extremal of 
an error curve will be either a true minimum or a true 
maximum. Possible exceptions occur only at end points of 
the interval of interest, in well-behaved cases. In general 
there will be one more extremal to an error curve than 
there are free parameters in the form. Thus, the two error 
curves given above each have four extremals, and there are 
three parameters (b^, b^, b^) in the form. Or looking at the 
matter in another light, there are seven extremals, and six 
parameters, as the approximation holds over ( — oo , -|- oo ) .* 
Thus, a final error curve results from the correct determi- 
nation of w -f- 1 extremal locations Ci or, more helpfully, 

aAnd odd power terms are missing. 
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from the correct determination of n extremal locations ei 
and one common deviation d, that is the absolute magnitude 
of the error curve at each of the n -\- \ extremals. 

While our problem is then accurately described as one of 
solving a system of transcendental equations in « -f- 1 un- 
knowns ei and d, actually our problem amounts to nothing 
more than that of solving a single transcendental equation 
in d. And this is quite a simple matter. In practice, the 
values of £?i* that we "read" from a given intermediate error 
curve are used unchanged in the cycle that follows. The 
drift of location is usually very small, and the results are 
somewhat insensitive to the ei*. (An abbreviated numerical 
process is used in lieu of curve plotting, and hence "read" 
is in quotes.) 

The Problem of Sensitivity 

In almost all cases of rational approximation, we run into 
a problem of sensitivity. Thus, a function f(x) may behave 
decently, have an adequate rational approximation of the 
form 



f (^) = 



N(x) 
D{x) 



(4) 



and yet the components N{x) and D{x) may behave very 
badly, changing in joint fashion by many orders of magni- 
tude as X ranges over the interval of approximation. For- 
tunately, however, there always seems to be a happy solu- 
tion to this kind of difficulty. 

Thus, in making a 4-decimal {d = .000139) approxima- 
tion to 



</>(^) = 



1 - g-^ 

X 



(5) 



over (0, 00 ), we cast the result into the desensitized form 
1 



i = 



<l>*ix) = 



1 -f .3606032.r 
.3671626 i 



(6) 
.2272232 e + .8601996 e 



1 - 1.3562710 i + 1.6148087 e - .2585377 f 



in which a Z)(l) = 1 condition has been irtiposed on the 
second form by proper choice of parameter in the first form. 
It is easy to see that (6) is an extremely tractable ap- 
proximation — what we shall call a computable approxima- 
tion. The variable | is limited to the interval (0, 1) of 
variation, all coefficients and individual terms in the ex- 
pression are of reasonable magnitude, and the denominator 
of the second form behaves very well. This may be seen in 
Figure 4. 

In very many instances, the least sensitive representa- 
tion that may be obtained is obtained through imposing a 
D(a) = D(b) = 1 condition on the denominator when an 
approximation is made over {a,b). 




1.0 



Figure 4 

The Problem of Best Fit 

Our criterion of best fit will be that of minimum devia- 
tion. Can we then say that our sinusoidal error curves give 
evidence, in each instance, that no better fit can be obtained ? 
We now give an interesting result for polynomial forms. 
The result is, undoubtedly, a familiar one to workers in the 
field of polynomial approximation. 



Consider the polynomial form 

y = ttyX^i -\- a^x'^'z -[-... + ClnX^n , 



(7) 



with n free parameters a^, and specified powers pi that are 
distinct integers ^ />i < />2 < . . . < ^w, and n -\- \ points 
(^i, yi), i = 0(l)w, for which < ^iTq < ^i < . . . < ^». 
We introduce the residual notation 



Ti = a.Xi^i + . . . + «w^/" - yi > 



(8) 



and let Ai denote the value of the determinant obtained 
from the w X w -f- 1 matrix 



.^0^1 , x^Pi , 









(9) 



by deletion of the Xi column. A brief argument will show 
that all Ai are positive. 

A proof is by induction on n. Expand any Ai by elements 
of the last column, and refer to Descartes' rule of signs. 
There are at most « — 1 roots to the polynomial in the last 
column variable, and these are each of the remaining col- 
umn variables. Our polynomial may then be factored in 
such a fashion that all factors are easily seen to be positive, 
and the result follows. 
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Now, we notice that minimization of 

n 



proximates the Gaussian error function 



(10) 



with respect to the a^ yields the equations of condition 

r,= (-i)*ro. (11) 

Let us denote the residuals in this particular case by the 
notation T*, and write 

T* = i n*| . (12) 

Let Ti** stand for the residuals that result from any other 
choice of parameters tti, and write 
Max 



M = 

*• 

It then follows that 



Ti** 



(13) 



n n n ■» 



t=0 t=0 



which says that T* < M. The result may be stated as a 
theorem. 

Theorem : The minimum deviation solution is such that 
all deviations are equated in absolute value, and the signs 
of the deviations alternate, unless the form passes exactly 
through the n -{- I points, in which case all deviations are 
zero. 

This result tells us, in the case of polynomial approxima- 
tion, that if we obtain an error curve with n -\- \ extremals 
that obey the conditions of the above theorem, then the 
approximation cannot be improved. For no other instance 
of the same form can approximate those particular n -\- \ 
points of the curve to be approximated as closely as the 
instance in question. 

Iterated Rational Forms 

Certain limiting cases of rational approximation may 
also be of practical interest. Thus, we may find a form of 
the type 

f(ax) = R[f{x)] , (15) 

in which R(>i) is a rational function of A, quite adaptable in 
special instances. Here a stands, in general, for a positive 
constant sizeably greater than unity. Quite obviously, the 
problem of fitting such a form is just that of making a 
usual rational fit. 



Thus, consider the function that satisfies 

h(^A = 3H^(x) - 2H'{x) , 
and has the power series expansion 

^^") =2+\V2^;~15W2^j + 975-W2^j+- 

about the origin. Here H(0) = 1/2 is a fixed point, and 
H'(0) is an imposed condition. H{x) rather closely ap- 



(16) 



.(17) 



''-' -fl^^ 



e-i' dt , 



(18) 



over ( — ^00, 00 ) as may be seen in the difference curve 
plotted below. Note that F{x) and H{x) both have the 
same type of odd symmetrical behavior, so that our differ- 
ence curve need but be given over (0, oo ), Figure 5. 




Figure 5 



Multivariate Approximations 

We are interested in the subject of a generalized empiri- 
cal analysis in which multivariate forms of a very general 
type are studied. These are non-parametric forms in which 
the unknown quantities are univariate functions of the 
separate independent variables. 

By considering basic functional forms, such as 






(19) 



we decompose the problem of multivariate approximation 
into its fundamentals. Here fiix) and giiy) are unknown 
univariate functions to be determined so that A* (.^,3;) shall 
approximate a specified function h{x,y) over a region of 
interest as well as possible. 

Thus, if the form (19) is inadequate, in an instance, no 
parametric form of similar structure can be adequate in the 
same instance. Considerable work has been done on the 
problem of determining required key values of the unknown 
functions in such forms. Once tables of values are available 
for each unknown function, the art of univariate approxi- 
mation may be employed to complete the fitting process. 

A further example of a general functional form is the 
slide rule form 

h{z*)=f{x)-\-g(iy) (20) 

in which 2* = z*{x,y) is to approximate 3(x,y) over a 
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specified region. Thus, in a large machine computation that 
we undertook for a division of RAND, we approximated 



B{x,y) = >J"^I^^(^l-;r<^-)^ (21) 

to 3 decimals over 1 .2 ^ y ^ 1 .4, .05 ^ .r ^ 1 by such a 
form, and prepared turning point tables of/(;i:), ^'(y), and 
h{s). These were used with efficient success. 

Of great practical interest, in the approximation of well 
behaved bivariate tables, is the simple form 

s''{x,y) = jx{x) g^{y) ^U{x) g^{y) + ...-\-jn{x) gn{y) . 

(22) 
To give an impressive example, for the reader to learn 
the power of such a form in the fitting of useful tables, we 
note that 



%J o %J o 



V(h,q) = i- 



e-l^'^^^v^^ dy dx , 



(23) 



may be approximated by 

F*(M) =U{h)g.{^^-^Uh)g,{^^ (24) 

over ^ /i ^ 00 , ^ q/h ^ 1 to 4 decimals. This suffices, 
with the aid of a table of the error integral, to yield all 
values of V{h,q). 

1. C. Lanczos, "Trigonometric Interpolation of Empirical and 
Analytical Functions," Journal of Mathematics and Physics, 
Vol. XVII, No. 3, September, 1938. 

2. H. H. Germond, "Miscellaneous Probability Tables," AMP 
Note No. 14 (unclassified). 

DISCUSSION 

Professor Runs: I am interested in how this scheme, 
which you use for getting a polynomial approximation, 
compares with using Tchebyscheff polynomials. Do you 
use that technique ? 



Mr. Hastings: We often use that method in fitting poly- 
nomials. 

Professor Kunz: The Tchebyscheff gives you the best fit 
in the same sense that you have used it here? 

Mr. Hastings: Not exactly. It gives it to first order. 

Dr. King: Dr. Tukey and I have a theorem that when a 
man gets associated with computing machinery he seems to 
spend a lot of time discovering previous fields of mathe- 
matics. The first part of your talk is really the theory of 
Fades' tables, for which there is quite an extensive litera- 
ture, although not one that one comes across very com- 
monly. I don't mean to detract at all from the value of what 
you have done, but to point out, rather, that there are exist- 
ing theorems. In fact, one of the difficulties with much of 
the mathematical literature is that it is not too useful when 
you want to put it on machines. 

In particular, I would like to point out that the poly- 
nomial approximation has one especially simple form ; when 
you try to approximate a function by a continued fraction, 
you get exactly that form, and the theory of the Fade tables 
was to take that particular form and adjust the coefficients 
in the polynomials to get a better approximation. In other 
words, a continued fraction is a relationship between the 
coefficients of the polynomial ; while the Fade theorem ap- 
proach was to try and investigate what are the best poly- 
nomials to use. I might say that, as far as I know, they 
never came to any good conclusion, either as to what you 
mean by "best" or how you get there ! 

Mr. Hastings: Thank you. 

Dr. Tukey: I would like to ask the speaker if he would 
agree with the following position : That the real purpose of 
computing a table is so that you can get a good approxima- 
tion and throw the table you just computed away. 

Mr. Hastings: I probably would keep the table, even if I 
did not use it. 
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THE FIRST IDEA I wish to mention is a very simple 
one that comes from calculus. This idea occurred to me 
years ago when we had only a 601 and were unable to per- 
form division. 

Suppose our purpose is to tabulate an analytic function, 
j{x) = X, at equal intervals of the argument, x, where X 
represents the numerical value placed in the table for each 
entry. This method is to be applied in the cases where there 
exists an inverse function, P{X) = x, and will be practi- 
cable, if it is simpler to compute /^(X) than it is to compute 
j{x) by some direct means. 

The problem may be stated as follows : If the series for 
sin <l> can be expanded on a program cycle or a series of 
programs, then if the inverse function is needed, the 
series is expanded until it equals the sine, thus giving the 
angle. If we write F{Xq -{- e) — x ~ and expand the 
expression by Taylor's theorem and use Newton's method 
X - F{X,) 



of approximation, e 



is obtained, where Xq 



P'iX,) 

is some approximation to the correct value. Also the follow- 
ing equations are true : 

dX = f{x)dx 

dx = F'{X)dx 

dx 



P'(X) 



= f{x) 



Now write 

X, = X, + ^ = Xo + [^ 



P{X,)] 



A! 

w 



(1) 



This is a first-order approximation, but it is applied only 
to the residual which is supposed to be small. If a set of 
approximate Xq's can be obtained for the entries to be put 
into the table, this iterative equation is computed instead of 
attempting to compute j{x). This is on the assumption that 
F(Xo) can be computed more readily than f{x). In the 
case of an arc sine table, for example, this means that one 
must have, necessarily, a table of sines to use, or a sine 
series, or something similar. 

If the table is to be at equal intervals of the argument, pre- 
sumably the entries are in units of a certain decimal place, 
so that 1/w simply requires proper placing of the decimal 
point, and A^ is obtained by differencing the values to be 



placed in the table at that stage. If necessary, a first ap- 
proximation can be obtained as illustrated in Figure 1. 




Figure; 1 



The true curve is approximated by a series of chords. 
Each value of X^ is obtained by summary punching with 
progressive totals, tabulating blank cards, and accumulating 
a constant first difference from the digit emitter. Some hand 
computing is necessary to find the points where the constant 
emitted first difference should be changed, and the number 
of changes is balanced against the size of the allowable 
error of X^. 

The division by F'(Zo) has been replaced by a multi- 
plication by A^, and equation (1) is iterated repeatedly 
until the final values are reached. Although this procedure 
may not appear attractive, it is for the purpose of eliminat- 
ing the division. It may be applied if a table, say, to four 
places is available, and a table to eight places is needed. 
Perhaps some of Mr. Hastings' inverse functions could be 
generated more easily than the direct functions. It would 
have an application there. 

Next, I would like to indicate the results in two sim- 
ple cases. If a reciprocal table is to be obtained, then 
f(x) = X = l/x, and f{x) is eliminated, as derived from 
this expression, instead of substituting A^/w in equation 
(1). Then Z = Xq -f- (1 — xXo)Xq. It is not necessary to 
summary punch any differences. An iterative formula is ob- 
tained which would have been obtained more easily some 



62 



SEMINAR 



63 



other way! In the second case, if j{x) = x'^'"^, the same 
procedure is appHed, obtaining 

X = Zo + (0.5 - ^.SxXl)X^ . 

This is a quantity which arises when transforming from 
rectangular to polar coordinates. It is necessary to divide 
by the square root of r^. 

Now I would like to say a few things about the construc- 
tion of a sine table. My remarks apply beautifully to the 
sine and cosine function, and contain some ideas which may 
be extended to other functions. Let Figure 2 represent the 
quantities in a sine table at equal intervals of the argument. 



(f) — w sin (<f> — w) A sin (<f> — w) . . . 

A^ A^ 

<f> sin <f> A sin <^ A- sin ^ 

A^ A^ 

^ + w sin (^-|-w) A sin (^-)-w) . . . 



Figure; 2 



The second difference opposite sin <j> is 

sin {<j) + w) —2 sin ^ + sin (^ — w) 

= —2 ( 1 — cos w) sin ^ = ^ sin <\> . 

Similarly, the fourth difference becomes A^ sin ^, which is 
rigorous, and not an approximation. This is a property of 
the sine function. The above suggests that if interpolation 
is desired, the best procedure is to use Everett's interpola- 
tion formula. This will reduce to : 

m ^^ — —, -A 



w(l- — m^) (2^ — m^) 
_ 

w(P — w^) n(V--n^){2 
■A -\ 



3! 
A'^ -\- . .. + sin (<^ -f w) n 



'A^' + 



•] 



(2) 



3! ' 5! 

This process of interpolating for a sine between two given 
values means that each of the values is multiplied by its 
corresponding square brackets, which is, in general, differ- 
ent from the ordinary concept of interpolation. 

It is seen from trigonometry that the square brackets in 
equation (2) which have been derived by means of working 
with Everett's interpolation formula, have closed expres- 
sions, namely 

sin (1 — n)w , sin nw 

■ — -^-: — and —. . 

sm w sm w 

Here I would like to point out something which is just a 
curiosity. Suppose ^ = 0, then the first line of equation (2) 
will drop out. Consider the second line. One of the first 
things the teacher tries to emphasize when this subject is 



reached in trigonometry is that sin nw is not equal to 
n sin w, but you see that, with the exception of the higher 
order terms in the series, it is true. So that is a good way 
to confuse everybody ! 

I shall describe briefly how we constructed an eight-place 
sine table, in what we considered a most efficient manner of 
arriving at a set of punched cards for the table. Ninety 
cards were key punched, proof read, and differenced, each 
containing the sine of an integral degree. If the argument 
is in degrees and if an eight-place sine table is desired, 
then the interval must be O.°01 in order to have linear in- 
terpolation. This yields a second difference of three units in 
the eighth place. Since the second difference coefficient is 
never greater than one eighth, the error is less than a half 
unit in the last place. 

If the expression in the square brackets of equation (2) 
was computed one hundred times for n = 0.01, 0.02, etc., 
and the results used to form the one hundred interpolates, 
the basic idea of the process can be seen. Now this complete 
set of square brackets is much like a sine and cosine table 
itself, because after going from n = 0.0 to 0.5 for the first 
bracket, the second half from 0.5 to 1.00 may be obtained 
by reading up the column, as one reads down one column 
to get the sine from 0° to 45° and then back up the cosine 
column to obtain values from 45° to 90°. 

In the present case it means that when one multiplication 
has been made of a square bracket times sin (f>, it is used 
once for a given value of m in one degree and again for the 
same value of n in the next degree. Although every single 
sine entry is obtained as the sum of two products, there are 
only as many products as there are entries in the table, be- 
cause each one is used twice. 

In practice the entries are not formed directly by equa- 
tion (2) but the square brackets are differenced, the prod- 
ucts formed, and then the interpolates are built up by pro- 
gressive totals over each range of one degree. This process 
enables a multiplication with eight-figure factors (the nor- 
mal capacity of a 601) and still protects the end figures 
against accumulated roundings. The differences of the 
square brackets are of the form : 

0.01 + AA^B, + A'A^B, -f . . . 
If this expression is evaluated to 12 decimal places, there 
will be only 8 significant figures beside the leading 0.01. 
A ^£2 means the first difference of the Everett second differ- 
ence coefficients. The multiplication of sin <f) by 0.01 is 
accomplished by crossfooting, and the rest is multiplied in 
the usual way. This allows two decimal places for protec- 
tion of the end figures, owing to the progressive totals 100 
times, and two extra places in the computation to insure 
the correct rounding to the closest half unit. 

Nine thousand multiplications were performed, using 100 
group multipliers, and the work was arranged with three 
fields to the card. Then the cards were sorted and tabu- 
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lated ; 3,000 summary cards yielded the final values for the 
table. There was an automatic check as each sine of an 
integral degree was reproduced at the end of that interval. 
The final table was then reproduced, and it was necessary 
to punch the first differences of the rounded values in order 
to interpolate these. The final check was the differencing 
of the first differences, which were inspected. The entire 
operation took about twenty hours. 

You will perceive readily, from Figure 2, that a sine 
table may be constructed simply by multiplying ^ sin ^ = 
A^ (sin <^) and building up step by step. The process has 
the disadvantage of accumulated rounding off errors, just 
as in numerical integration ; thus, more places must be car- 
ried as a protection. In fact, this is the process of numerical 
integration in the case of this simple function. That leads 
me to make one other comment : by means of numerical in- 
tegration it is possible to tabulate these functions, or to 
generate the functions so that cards are punched. 

There is no need to discuss the subject further, since it 
is all covered in the subject of numerical integration. The 
ease with which equations can be integrated depends upon 
how much protection can be obtained against the accumula- 
tion errors, and how well the integrands, needed for the 
integration, can be computed. 

The; above; concludes the remarks with respect to tables, 
which have equal intervals of the argument. The remaining 
remarks will apply to optimum interval tables, and I am not 
sure whether or not it is necessary to describe the essential 
property of an optimum interval table. It amounts to this : 
First, consider the simplest way of interpolating a table 
linearly: /„ = /^ -)- wA^. Of course, in all the tables that 
everyone uses from high school on, the interval of the 
argument, w, is in some unit in the decimal system and one 
never thinks of the whole role that is played by the frac- 
tion n. It is necessary to consider what happens if the in- 
terval has some odd value, for example 0.2. Actually n = 
(x — Xq)/w and we shall write our interpolation formula 
in the form 

/. = (/.-^A') + ^(^) = F. + ^D. , (3) 

What does this mean in numbers? This is illustrated in the 
following example : 



■^0 

12.0 
12.2 
12.4 
12.6 



A^ 



222 
224 
226 



0.17318 
0.17316 
0.17312 



1110 
1120 
1130 



/o 

0.17318 

0.17540 

0.17764 

0.17990 

If ;ir = 12.325, /(.;i:) = 0.17316 + 0.325(1120) = 0.17680. 
It is obvious, at a glance, that since 0.325 lies between 0.2 
and 0.4, one is not supposed to multiply directly by the 
digit 3. There must be something wrong here. If we had 
applied this operation to the /o and A^ columns, it would be 
wrong. But the Fq and D^ columns have been adjusted so as 



to compensate in advance for the error we would otherwise 
make. This means that all the card columns corresponding 
to .325 are wired directly to the multiplier unit and the 
machine takes no cognizance of whether the interval is or- 
dinary or unusual. 

That is all there is to an optimum interval table. You see, 
you fool the machine by giving it a table which doesn't make 
sense. The control panel is wired in a way that doesn't make 
sense. Generally, this theorem doesn't hold, but in this case 
the two nonsenses make sense. 

What I would like to show you is a very unsophisticated 
way of looking at the construction of such a table, if a table 
with second order interpolation is desired. The generaliza- 
tions are fairly obvious. If one wishes to have an interpola- 
tion formula of the form 

fn = Po + ND, + NW, (4) 

the way in which we shall approach the problem is as fol- 
lows : Suppose the intervals of the argument, which we are 
to use, have been established, the interval may be changed 
whenever the function's variation warrants, but within the 
restriction that the combinations of intervals must always 
end in a cycle of ten units of the left-hand position of the 
multiplier N. Thus, you may have combinations like 0.2, 
0.3, 0.2, 0.3 or 0.3, 0.3, 0.4 or 0.5, 0.5. We shall call the 
interval length 2a and the value of the argument at the 
center of the interval Xq. Now expand the function in a 
Taylor's series about the mid-point. In the present case, 
instead of trying to write down completely general equa- 
tions, I shall write down the results as if we are interested 
in constructing a table oi f{x) = x~^^^. This is a table that 
is needed repeatedly in dynamical astronomy because we 
have 1/r^, which is the law of gravitation, and another 
factor x/r, which is the projection onto the x-axis (and 
similarly for 3; and ^). Thus, r^ always will be in the denomi- 
nator and it is easy to obtain r^ = x^ -\- y^ -{- z^. We then 
have: 

j{x) = x-^'\ 



f"(^) = + 
f'ix) = - 



-5/2 _ 



3fix) 
2 



15 
4 
105 



r-7/2 ^ 



+ 



x-^/^ = 



945 



X 

15 fix) 
4 x^ ' 
105 /(^ 

8 x^ ' 
945 fix) 

16 x^ 



fix) = fiXo + h) 



_ J-fl --Ail 4-MAl 

- xrl 2x,Y^2Axl] 

+ MAii + 63/^) n 



The terms h'^/x\ in the braces are actually the third and 
fourth derivative terms which cannot be included because 
the interpolation formula is to be only quadratic. However, 
since these terms are always positive, they shall be used as 
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Figure; 3 

illustrated in Figure 3. I^et B be the total error committed 
Sit X = Xq -{- a ii we neglect the third order term com- 
pletely. Let KB be the fractional part of this error which is 
taken into account if the cubic term is replaced by a linear 
term, as shown. Then the remaining neglected error is 
h^B — hKB. This error has a maximum (shown by the 
short vertical line) at /i = \/K/3, If the error is set at 
this point equal in magnitude and with opposite sign to 
the error at the end of the interval, we have 



3\3 



K 



)- 



{K - l)BandK - 3/4. 



If we analyze the quartic term in the same way, we ob- 
tain K = 2( V2 — 1). Our interpolation formula becomes 



«^)=:^['-|J;{'H-!i|}'' 



+ 



15J_ ( 
8K\ 



1 + 



(V2-l)63 



1 



63 aM , ,"1 



Since the two braces are so nearly alike, we may use, with- 
out sensible error, 1 -|- 1.09375 a^/xl for each of them. 

Now we may expect that the interval which may be 
used is somewhat more favorable than that which would 
be determined on the basis of neglecting the third and 
fourth order terms completely. I shall let Dr. Grosch ex- 
plain the way in which the intervals are obtained. What is 
done as a rule of thumb is to write 192^ = w^f"(:r), where 
e is the admissible error, usually one-half unit in the last 
place. Then the size of the third derivative will control the 
value of the interval which may be used. 

At this stage we have 

f(^)=fo-^hh-\-h%. (5) 

We are still faced with one other problem before we are 
finished: h is counted from the middle of the interval; so 
we shall write n — a = h. Then n is counted in the same 
units as h, but from the beginning of the interval. But if the 



value of the argument at the beginning of the interval is not 
zero, but A, then let AT — ^ = n where N is the number 
which is actually used as the multiplier in equation (4). 
Making these substitutions in (5) in order to reduce it to 
the form of (4), we find that all of the following relations 
exist. I shall write down only the end results. 
D,=h, D, = f^-2{a + A)D, 
^ i^o=/o- (a + A)D,-(a + AyD,. 
This is about the simplest way, of which I could think, to 
present the development from the function and a Taylor's 
series to the final results entered in the table. 

DISCUSSION 

Dr. King: I would like to make some much more general 
remarks. It is a good thing for a lot of people to work on 
these problems so as to make sure that the best method 
finally comes out. 

On the other hand, there is a point where it is inefficient 
to have too many people, and I would like to ask the speak- 
ers whether they think the last word has been more or less 
said on optimum interval tables, and, if so, I am sure there 
are some particular little details that could be improved. 
So I would like to hear from them whether they think the 
time is ripe for people to get together and have one system 
of optimum interval tables. 

Dr. Grosch: I think, honestly, we can say that the poly- 
nomial case for the single variable is just about under con- 
trol now. By the time you go to two variables it becomes 
so complicated that it may not be worth investigating 
until we have some big bi- or trivariates that we just have 
to make. It is really a beastly job, even in the linear case. 
I have made some explorations in that direction and don't 
feel at all satisfied. In the univariate case, I don't think 
there is much we can do beyond this inverse matrix busi- 
ness, and the reason I am so sure of it is this : that if you 
pick any interval (and you may pick a wrong interval, be- 
cause several terms in the Taylor series are contributing; 
higher order terms, as Dr. Herget shows, are being added 
with lower order terms and so forth) ; but if you pick an 
interval under any assumption whatsoever, Mr. Hastings' 
comment of yesterday is the key to the whole situation that 
the error curve for a certain degree of approximation is 
going to look just about the same. It will change a little bit. 
He said Tchebyschefif was the zeroth order approximation 
to that curve. It will change a little, but the position of the 
extrema is very stable. Therefore, you are going to make 
an error of e at the position where you think those extrema 
are going to occur ; and, even if the function doesn't be- 
have quite the way its next higher term indicates it should, 
the extrema aren't going to shift very much. Therefore, 
your value of the actual error curve obtained when you 
use the table will not be more than a tenth of a per cent 
or a hundredth of a per cent greater than theoretical t. 
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unless you come to a curve so very bad that the error 
curve doesn't look anything like a Tchebyscheff polyno- 
mial; and, of course, we can always invent such curves. 
But I think they are going to be quite hard to invent. 

I also expect that the rational function is going to have 
a very stable error curve, what Professor Tukey referred 
to as the Tchebyscheff polynomial for rational functions. 
But I don't have that as yet and I don't know whether 
Mr. Hastings has. 

Professor Kunz: I think one of the important things in 
this talk is that Dr. Grosch has reminded us that there are 
other ways of interpolating. Just as a very trivial sugges- 
tion: if you take a hyperbola and pass it through three 
points, this gives you second order interpolation in a more 
general sense. Some of you who haven't tried similar 
things might like to try it. One of the interesting prop- 
erties is that you try inverse interpolation with such a 
function, and it is just as easy as direct interpolation. You 
can obtain second order inverse interpolation very nicely. 
I have used this in quite a few cases, and it sometimes 
yields a very nice fit to curves, particularly if they have a 
singularity somewhere in the region. 

It is just a suggestion to become sort of initiated to 
these reciprocal differences which are a little forbidding 
and are awfully hard to integrate. 



Professor Tukey: I cannot agree with Professor Kunz 
in the use of the word "interpolation." The essential point 
about this is that we have given up interpolating, just as 
we have given up expanding in series. We are trying to 
get something that is a good fit, and that is a different 
problem. 

Mr. Bell: While we are on the subject of tables, I would 
like to point out another way of getting a fit to curves of 
various sorts. It is a widespread opinion among engineers 
that a problem which involves curves of some sort cannot 
be done on punched cards. I am talking, of course, about 
engineers who have hearsay knowledge of IBM equip- 
ment. 

Now, this is not true. All you have to do is read a lot 
of points. With the points you can obtain first differences, 
set up a linear interpolation, which can be done quite 
quickly. Of course, this is completely non-elegant, but 
very practical. Many times you have whole families of 
curves. We, in our organization, are fortunate in having 
rapid ways of reading such data. We can read, say, a 
thousand points from families of curves in maybe an hour 
and be ready to go on a problem without mathematics. 
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THE SEVERAL optimum interval tables included in 
this paper were constructed for linear interpolation on 
the IBM Type 602 Calculating Punch in an endeavor to 
accelerate the process of table look-up. In each case a 
critical table of thousands of lines was reduced to a table 
of fewer than two hundred lines. The number of lines per 
table might have been still further reduced by using a 
higher order of interpolation, but this was not desirable 
since the interpolating time on the type 602 calculating 
punch is approximately proportional to the order of inter- 
polation. 

The following tables were constructed: 



Table Function Accuracy 



Range and Interval Size 



A 



B 



1 • 10-^ 



1 • 10-^ 



t= -1.7000(.0114 
- .0102) -0.4000 104 cards 



1_^V7 1 . 10-5 



!^ r = .30000 (.00350 
(l-r^/Q^ 1 • 10-^J - .0001 3). 99900 192 cards 

C Arc cosh .«• 1 • 10-* X = 1.0002 (.0001 

-1.0500)27.3600 132 cards 

Tables A and B each consist of two functions with a com- 
mon argument. This arrangement was convenient in that 
both functions of a given table were needed, at the same 
time, in the particular problem for which the table was 
constructed. Only one sorting operation is necessary to file 
the table cards with the detail cards in preparation for the 
interpolation of both functions. Including two functions in 
a table with a common argument may result in a slightly 
larger number of lines than would be obtained in either of 
the functions if each were optimized independently. How- 
ever, the additional lines are of little consequence, since an 
entire sorting operation is eliminated. 

The tables were constructed using a method developed 
by Dr. Herbert R. J. Grosch of the Watson Scientific Com- 
puting Laboratory. 

The method consists of dividing the interval {a,h) upon 
which the required function is to be approximated, into a 
number of sub-intervals, upon which the function is re- 
placed by straight lines of the form (Figure 1) 

j{x) =hi-\-mi{x-Xi) . (1) 
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For the functions included in this paper it is convenient 
that hi be referenced to the vertical axis at Xi for each inter- 
val, thereby limiting its magnitude. The optimum sub- 
intervals are determined by the expression 

4Vi 

(2) 



ill* 
dxH 



where 



.the tabular interval 



~^ = the 2nd derivative of the function, f{x) 

c = the maximum theoretical error between the ap- 
proximate value and the true value of the function. 
The number of lines. A'", can be found approximately 
from the expression 

N{a,h) = / ^. (3) 
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Since each of the tables was constructed in the same 
manner, a description of Table B will illustrate the details 
of construction. 

This table was constructed such that the functions are 
everywhere represented on the interval to an accuracy of 
1 X 10-° (total error). 

The total error consists of the sum of the theoretical 
error (e) between the straight line and the function, the 
rounding error when interpolating (1), and the departure 
of bi from theory due to rounding. Hence, for each of the 
two functions 

fir) = 1 - r'/' (4) 

(1 - r'-/')h 



Fir) = 



y.2/7 



(5) 



the value of e is as follows : 

1 X 10-= = £ + .5 X 10-' + .05 X 10-=, 

c = .45 X 10-^ (6) 

It will be noted that the inclusion of "the rounding error 
when interpolating (1)" insures that the rounded inter- 
polated value of the function agrees within 1 X 10"= of the 
true unrounded value of the function. 

An examination of the second derivatives of the functions 
discloses the direction of increasing w, i.e., the successive 
values of w increase as the absolute values of the derivative 
decrease. For the function fir), the intervals increase from 
left to right throughout the range of the function. However, 
for Fir) the intervals increase from left to right to the in- 
flection point, at r = .73115, for the left-hand portion of 
the curve, and from right to left for the right-hand portion 
of the curve. Hence, for the function Fir) the two sections 
of the curve are treated independently and the intervals 
calculated from r = .30000 and r = .99900 toward the in- 
flection point. 

The second derivative of f{r) and the value of c when 
substituted in (2) give 

a> = 18.782971 X 10-^ r«/^ (7) 

Beginning at r = .30000 and substituting in (7), the 
first interval was computed. The interval is added to the 
value of r to establish the new r for calculating the next 
interval. The process is repeated through r = .99900. Each 
value of 0) is calculated to seven places, but the last two 
places are dropped and the unrounded five-place value used 
to establish the starting point for the next interval. 

The second derivative of Fir) and the value of c when 
substituted in (2) give 

_ 59.396970 r^/^ (l-r^/QS X 10-^ 
(8r*/^ - 27r'^' - 18)^ 

The intervals were then computed for each portion of the 
curve. As the value of r approached the inflection point, the 
intervals increase and become infinite at the inflection point. 
The intervals which cross the inflection point are shortened 
to end at that point. 

A comparison of the intervals for the two functions dis- 
closed that the intervals for Fir) were smaller than for 



(8) 



fir) over the complete range, and the controlling factor in 
establishing the tabular values of r for both functions. Then 
the values of the functions for each argument were calcu- 
lated with seven-place logarithm tables. 

The final step in the preparation of the table involved the 
calculation of the interpolation coefficients for each interval. 
This was accomplished on the type 602 calculating punch. 

The straight lines which approximate the function, fir), 
have the particular form 

fir) = h- Mi (r - ri), Vi^r < n^^ 
where /(r.)-f(n..) 



= tan (Xi = 



^i+1 ri 



and 

hi = fi — f. . 

Analogous formulas hold for the function Fir), except 
that Bi = Fi -\- e for the right-hand portion of the curve, 
since this portion of the curve is concave in the opposite 
direction. 

The values oi r, fir), and Fir) were punched in a deck 
of cards and the values Wi and bi for fir), and Mi and Bi 
for F(r) calculated in two passes of the cards in the calcu- 
lating punch. The values of bi, Bi are rounded to six places 
and the values of w^, Mi rounded to seven places. Only the 
five-place value of r, six-place values of bi, Bi, and seven- 
place values of mi. Mi are used in the final table deck for 
interpolating. 

In practice the table cards are sorted in front of the de- 
tail cards on a common field r. A single interpolating con- 
trol panel calculates the values fir) on one pass and Fir) 
on a second pass. The second pass is preceded by an 
x-punched lead card which controls the reading and punch- 
ing in the proper fields. 

Three man-weeks were required to construct the table, 
but a five-place critical table of 69,900 cards was reduced 
to an interpolation table of 192 cards. Less time was re- 
quired for preparation of this table than to key punch and 
verify a critical table. Use of the interpolation table greatly 
accelerates the process of table look-up. 

DISCUSSION 

Dr. Her get: Suppose that we are going to have a six- 
significant-figure table. There is no reason why still an- 
other significant figure cannot be added in the first term, 
for the purpose of avoiding the round-oflf error from this 
particular term ; and, in addition to that, include an extra 
five, which is the half adjust for rounding. 

Also, when these intervals are estimated, another method 
can be used. Take such a function as was shown ; it was 
obviously easier to compute that than to compute any of its 
derivatives. Next, compute the function at fairly large equal 
intervals over the whole range and obtain the second dif- 
ference. These numbers, divided by the square root of the 
reduced interval, give an estimate of the second derivative 
times the square of the interval. 



Table Interpolation Employing the IBM. Type 604 
Electronic Calculating Punch 
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MUCH OFTHEWORK done in the tabulating sec- 
tion of the U. S. Navy's Operations Evaluation Group in- 
volves small numbers of cards. Most of the jobs also require 
table look-up of such common functions as exponentials, 
logarithms and the trigonometric ratios. Often, however, it 
is necessary to process from 10 to 90 times as many master 
cards as detail cards, in seeking these functions, when em- 
ploying the conventional table made up of arguments at 
equal intervals. The programming feature of the IBM Type 
604 Calculating Punch has introduced the opportunity of 
interpolating intermediate values lying between entries of a 
table, and, of course, each person has his own favorite tech- 
nique. Many systems of interpolation have been developed, 
and each appears to have advantages over others. 

The objective of punched card tables, it seems to us, is 
one in which there are very few entries in order to reduce 
card handling time, and in which sufficient accuracy can be 
developed. In an attempt to reach this objective, we have 
developed a method of table construction and table inter- 
polation which lends itself ideally to the small volume tabu- 
lating installation fortunate enough to possess a 604. Spe- 
cifically, the OEG has developed abbreviated tables which 
permit accurate interpolation up to 8 digits. 

Since this paper is on the application of punched card 
techniques to a mathematical problem, rather than an ele- 
gant development of a mathematical formula, the mathe- 
matics leading up to the formulation of the expression, used 
as a tool in this system, will be glossed over quickly. 

The table chosen for illustration is the natural log table 
with the range of argument from 1.000 to 9.999, inclusive. 
Conventional equal interval argument tables for this func- 
tion employ 9,000 cards, yet this method develops a file of 
but 327 cards. The conventional table can be of any accuracy 
required ; the one developed is correct to 8 X 10"^- 

Mr. E. B. Gardner, in an unclassified memorandum to 
the director of research, OEG, has developed a three-term 
formula for the interpolation of unequal interval arguments, 
which is nicely adaptable to the 604. Essentially, this proc- 
ess is one employing a constant (pre-selected) difference 



between successive values of the function, rather than be- 
tween the arguments. A formula based on the average of 
the Gauss "forward" and "backward" interpolation for- 
mulas, using divided diiTerences, was developed. 
The formula developed was : 

Ux = A^Bx-\- Cx^ , 
where Ux is the function of U of the argument x, and 
where m <_ I '^ x ^ a <_ b and 



A = 



B 



C = 



Ui+Ua I -{-a 



2 

AUi - {a-\-l)C 



AUi-\-alC 



AUi = 



and 



A'Ui = 



Ua - Ul 

a — I 

AUg- AUl 
b - I 



Once the initial interval between the values of the function 
has been determined, and the data recorded in some form 
for key punching, the cards were key punched, showing 
only the entry and the function of that entry. For ease in 
identification, these two fields will be called 1 and 2, re- 
spectively. 

The detail processing of the master cards follows step- 
wise: 

1. "Forward" gang punch fields 1 and 2 into fields 3 and 4. 
This process of "forward" gang punching, possible on 
the 521 punch, is a process which formerly required a 
reverse sequence sort before running through the con- 
ventional gang punch. This process transfers the infor- 
mation from a second card, into the first. 

2. Compute, on the 604, AU, as shown in the formula 
above, punching the result into field 5. 

3. "Forward" gang punch field 3 to field 6, and field 5 to 
field 7 (see card form). 

4. Compute A^U, punching the result in field 8. 

5. "Backward" (conventional) gang punch A~U into 
^ field 9. 

The card layout now contains the following information : 
Field 1234 567 8 9 



/ Ul 



Ua AUl b AUa A'Ui A'U„ 



69 



1^ 
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which is all the data needed to compute the three terms of 
the interpolation formula. 

In three passes on the 604, C, B, and A are computed, in 
that order, punching the results as C 12.34 X 10^^ , 

B 12345.67 X lO"' , 
A 123456789. X lO"* . 
It is not necessary to show the exponent of 10, which has 
been introduced only to keep the decimal point on the card 
during programming, and is not actually used in the com- 
putation. 

The usual steps of sorting and merging are employed to 
place the detail cards behind the proper master card (equal 
or lower), on the basis of the argument. The detail card 
argument, a five-place number, is merged against the entry, 
on the basis of the first four digits only. The value of the 
function to the fifth place of the argument will be inter- 
polated. 

The problem posed in the interpolation process was, pri- 
marily, a storage problem: 20 digits from a master card 
must be read into the 604, used, and held until the next 
master card was read. 

The storage grouping finally employed was : 

i^.S'1-2, the first (left-hand) 8 digits of the factor A. 

FS3, blank, for use during the computation of the inter- 
polated value. 

^►94, the 4 digits of C, and, in the right-hand position, the 
units position of A. 

GSl-2, the seven digits of B. 

G'5'3-4, for use during the computation of the interpolated 
value. 

MQ, for the argument, X. 

Program step: 

1. FS4.ro to arplus. 

2. Counter R&R from 2nd to GS3-4. This pair of steps 
eliminates the A^ digit. 

3. Multiply GS3-4 by X, producing CX. 

4. RO Ctr from 3rd to GSZ-4, retaining seven digits. 

5. RO GS2>-4 to the Ctr 3rd, subtracting, to leave a 
residue of the 8th and 9th digits of CX in the counter. 

6. RkR Ctr to PSZ. 

7. Multiply FSZ by X, producing the partial 8th through 
14th digits of CXK 

8. RMi. from 5th position of Ctr to i^,S*3, thus dropping 
decimals which have no effect on the final CX^ cal- 
culation. 

9. Multiply GSZ-4 by X, producing the 1st through 12th 
positions of CX^ (partial from 8 through 12). 

10. RO FSZ into 3rd Ctr, completing the value of CX^, 
digits 1-10. 

11. RO from 5th position of Ctr to GS2>-4 (digits 1-8 of 
CX^). 

12. R&R from 2nd position of Ctr to FSZ (digits 9-11). 

13. Multiply GSl-2 by X, producing BX (digits 1-12) . 



14. RO GSZ-4 into 5th of Ctr. 

15. RO FSZ to Ctr 2>rd. 

16. RO FS\-2 {A, digits 1-8). 

17. RO FS4 through shift read into 3rd, RI into FSZ. 
Note : This shifts out the unwanted values of C which 
were to the left of the 9th digits of ^. 

18. RO FS3 into counter. 

19. 1/2 adjust. 
Punch from counter. 

Sign control is necessary because A and/or B and/or C 
may be negative. The sign of A is carried in the factor stor- 
age sign hub 2, and controls normally the first 8 digits of A. 
The sign of B is carried normally in general storage 2, and 
operates normally. The sign of C is carried in the factor 
storage 4, and operates normally, although it introduces a 
difficulty in processing the sign for the 9th digit of A which 
is also carried in the same storage unit. If a negative sign is 
punched with an x punch, and if the master cards are also 
so punched, the solution is possible. By wiring the sign con- 
trol columns for both A and C, in addition to the wiring 
indicated above, from first reading brushes, through column 
splits to the transferred points of a punch selector under 
the control of a master card indication punch, thence to the 
I pickup of two pilot selectors, the basic control is estab- 
lished for the sign control for the first detail card following 
the master card. By matching signs, as in multiplication, a 
third pilot selector may be picked up when, and only when, 
the 9th term of A is to add, as shown in Figure 1. 

When this third pilot selector is normal, A^ is to subtract. 
In order to eflfect this control on detail cards other than the 
first of a group, the 2nd reading brushes covering these 
same control columns are read, because the signs from the 
master card have been gang punched as the detail cards pass 
the punch station. 
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An Algorithm for Fitting a Polynomial 
through n Given Points^ 

F. N. FRENKIEL H. POLACHEK 

Naval Ordnance Laboratory, White Oak, Maryland 



THE PT^OBLEM of fitting a polynomial through n 
given points has numerous applications in mathematical 
physics and engineering. In particular, this problem has 
become of considerable importance in the field of high-speed 
computation techniques where polynomial expressions are 
commonly used to replace any prescribed functions for in- 
terpolation purposes. The well-known Lagrangian inter- 
polation formula, of course, can be used for this purpose. 
However, the expression obtained in this manner is not in 
a form usually suitable for computation, since it consists of 
a sum of products, each of which involves the variables at 
the known points. In this paper we seek a simple procedure 
or algorithm for calculating (once and for all) the coeffi- 
cients a^, a^, a^, . . . ,an of the various terms of the polyno- 
mial which is given in the form, 

y = a^-\- a^x -\- a^x^ -\- . . .-\- an x"^^ (1) 

and which passes through n arbitrary points. 

To make the results more accessible to the reader, we 
shall first state (without proof) the principal expressions 
used in our algorithm. Then we shall illustrate the use of 
this algorithm for a typical problem, and finally we shall 
give a mathematical derivation of the basic equations. We 
shall use abbreviated notation for certain familiar functions 
involving the n letters Xi, x^, x^, . . . , Xn. We state these 
here. 



Definition i 



1 



{Xk- X^){Xtc- X^). . .{Xu- XTc-^){Xk- Xj,^^). . .{Xu- Xn) 



where 

,p-= 1 

Definition 2 

Elementary symmetric functions, p\, on the n letters, 
x^, X2, . . . , Xn taken i at a time. 
Definition j 

Elementary symmetric functions, fc/>" , on the (n—1) 
letters, 

jiTi, X2, . . . , X]c-i, .-Tfc+i, . . . , Xn taken i at a time. 

*This project was sponsored by the Office of Naval Research. 



Definition 4 

Sum, qf , of all possible distinct products, 
.«-f 1 x^2 . . . x^» such that a^ -f- aj -f . . . + a„ = i 
(a; = positive integer or zero), i.e. 

g'2 = -^i ~r ■^1-^2 "T -*^2 

Qii = X{ -\- X2 -J- X^ -f- X-^X^ -\- X-^X^ -\- X2XQ 

^S — ■^l 'T ^2 ~\~ ■^1-^2 + •^2''^1 

q^ = xl 

ql = pl = ^^ 4- ^^2 -h ^3 + ^4 + ^5 • 



Basic Relations 

Given n points (jr^, y^^), {x^, 3'2), • • • , (•*■«, y^), it is re- 
quired to obtain expressions for the coefficients a^, a^ , . . . , 
an of the polynomial ( 1 ) , which passes through these points. 
In terms of the functions defined above, these expressions 
may be given as follows 



«i = iP^y^ - g}«2 -?2«3 - ?3«4 - 

«2 = ip^y^ + 2p^y2 - ql^s - ql^, - 
«3 = iP^yi + 2p^y2-i- sP^ys- ql^^ - 



di = ip'yi + 2p% + . . . + ip'yi - q\au 



qn-i^n 

2 „ 
qn-2^n 

qn-s^n 



(2) 



qn-i^n 



an = i/'^yi + 2^''3'2 + • • • + nP^'yn • 

It will be noted that in these expressions only an is given 
in terms of known quantities, while a„_i may be computed 
in turn from known quantities and a«; a„_2 from a„_i and 
an ; etc. However, this can be carried out at the very end 
with very little additional work, after all the coefficients 
have been determined. The main job is to compute the p's 
and q's. We shall illustrate how this can be done systemati- 
cally and with relatively little computational work for the 
case of a polynomial passing through seven points. From 
this example the general procedure for n points will be 
apparent. 



71 



72 



COMPUTATION 



Illustrative Example 

We will apply our algorithm to the points x^ = 1, 



X, = 2^ 



, Xj = 7^; we will not fix the values of 



yi> y2> • • ' , yi- Substituting in equations (2), we obtain 
Oi = iP^yi - ql^z - q\a^ - ql a^ - qla^ - ^X - ql^y 

«2 = i/''3'i + 2p^y2 - qWs - ql^i- ^X - ^X - ql^^i 
«3 = iP^yi + 2p^y2 + zP^ys - q\<^i - qt^s - ql^e - ql^^i 
«4 = iP^yi + 2p^y2 + sP^y^ + iP^y^ - ^i^s - qX - qt^i 
^5 = iP^yi + 2p'y2 -i-sP'ys + 4/''y4 + ^P'y^ - ql^^ - ql^, 
fle = ip^yx + 2/'^y2 + zP^y^ + 4/''3'4 + yy^ + 6/'*3'6 - 3?«7 

^7 = l/''3'l + 2/''3'2 + Zp'yz + 4/''3'4 + ^P'y^ + 6/''3'6 + z/^'^T 

The computation of the p's is straightforward. We can 
compute them column-wise. Thus, for instance, 

i/'^ = 1 » iP^ = , iP^ = 7 — — — w _ — r,etc. 

X-i X2 1,-^1 X2) yX^ x^) 

For the computation of the q's the following procedure 

based on the relation (which will be proved later) 

ql = ^t qlx + q'r^ (3) 

should be used. First compute the ^'s in the first row, i.e. 



x^ = 1 ,ql = xl = I ,...,q^ 



x, = L 



Secondly, compute the q's on the first diagonal (containing 

q's), i.e. 

ql = ^1 = 1 , ?i= A'l + ^2 = 5 , . . . , 

ql = x^ -\- X2 -\- A-g -\- x^-\-x^-\- x^ = 91 . 
Finally, compute all other values of q by use of equation 
(3) . Thus the remaining q's in the second row may be com- 
puted, in sequence, as follows : 

ql = ql + •*'2 ?i = 21 
53 = ^3 + ■'^2 ql = 85 



ql = ql + ^2ql= 1365 . 

The ^'s in the third row are given by 
q% = ql + x,ql=U7 

ql = ql + ^3 ql = 1408 

9^4 = ql + ^z ql = 13013 . 
Similarly, the q's in all the succeeding rows may be obtained 
by this method. The values of the various coefficients for 
the above problem are given in the table below. 



Proof of Basic Relations (2) and (3) 

We shall now proceed to derive our basic relations (2) 
and (3) upon which our algorithm is based. We begin with 
the well-known Lagrange's interpolation formula which 
may be written in the form 
3'i = iP"" [ix-X2){x-x^) ... (x-Xn)] yi 

+ 2/^" [(^— 'i^i) (^-^3) . • • {^-^n)]y2 

+ ••• + «/''* [{X-X:,){X-X2) ... (x-Xn-i)] yn ■ (4) 

An obvious method of transforming Lagrange's equation 
(4) to the form of equation (1) is to multiply each of the 
above products and sum the coefficients of the like powers 
of X. This results in the following expressions for the co- 
efficients fli, a^, . . . , an- 

( - 1 )«-! a, = .^'^ ,PU yx + 2/'" 2PI.1 3'2 + . . . + «/»" nPU yn 
( - 1 )«-2 a, = ,/>« ,p^^^2 yi + 2/^" 2pn^ ^2 + . • . + np" npl.2 ^n 



( - 1 )'^ a, = i/>» ^pl.i y, 4- 2/'" 2pl.i 3'2 + • . • + «/»" np'U yn 



(5) 



+ a,_2 = ,p\pl yi -F s/'" 2ply2 + • . . + «/>" npl yn 

( - 1 )a,_, = ,/>» ,Pl y, + 2/'« ,p\ y, + . . . + n/'" nPl yn 
-\-an = .p"" yi + 2/'" ^2 + . • • + nP^'yn 

These relations differ from equations (2) inasmuch as 
the coefficients are given directly in terms of x^, x^, . . . , Xn', 
yi, y2> • ' ' } yn- From equations (5) it follows that relations 
(2) will hold if we can demonstrate that 

( - 1 )«-^ ai = ,/)" ,pi:i.i y, + 2/'" 2pl-i y2 + '-' + nP'' npZ.i yn 

= iP'yi + 2p'y2 + ----\-iP'yi 



+ ( - 1)"^ qi (,r l/'^i-l yi + --' + nP" nPl-i-r yn) (6) 



+ qU-t W .PI y. + 2/^" 2PI ^2 + . . . 

+ »/>" npl yn) - qi-i (x/'" yi + 2/'" ^2 + • • • + «/>" yn) 



x^ = \ a^ = 

X2 = 4 a^ = 

^3 = 9 a^ = 
X4 = 16 a^ = 

X^ = 25 ttg = 

x^ — 36aQ = 

x^ = 49 a^ = 



■.yi + 



a, — 



y2 - 



ag - 
5a, — 



24 
1 
360 
1 
8640 
1 



^1 - 



^1 + 



yi 



1 



15 

1 

180^^ 
1 



3'2+ 7K 



3780 
1 



^2 + 



1 

40 
1 
280 
1 
4480 
1 



^3 — 
3'3 + 
^3 - 



a^- 
21a^ — 

I4a^- 



a^ — 



85ag - 



147a, - 



1 



1260 
1 



11340 
1 
226800 ^* 



3'4 - 

^4 + 



30a, 



1 



72576 
1 



^5 - 
^5 + 



341a6 — 

1408a6- 

627 a^— 

SSa^ — 

^6 - 



»7 

1365a7 

13013a, 

11440a, 

2002a, 
91a, 



302400^' "^ 120960^' 120960^=' "^ 226800-^* 798336-^' ' 6652800 

1 1 1 1 1 1 1 

145 1 5200 ^' 5443200 ^' "^ 4838400 ^" 7484400 ^* "^ 19 160064 -" 86486400 ^' ^ 889574400 ^' 
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On the other hand, expression (6) will hold if we can prove 
the following two sets of identities 

(In-i kpi Qn-d+i) ~r kp2 Qn-(.i+2) • • • 

( - l)-(-i) ,/,»_(,,,, qi + ( - l)'^-^ ,,p"..i 

with k = 1,2, . . . ,i (7) 

Qn-i — kPi Qn-iUi) ~r Jcp2 ^»-(i+2) • • • 

k=l,2,...,n. _ (7') 

This is accomplished by carrying out the expansion of 
the ratio 

(X—X:i){x-X2) . . . {x-XTc-:L){^ — ^'k^i) • • • (x — Xn) 
(x — x^) (x — x^) . . . (x — Xjc) . . . (X — Xi) 

y-^ ^i+l ) • ' • \-^ ^n) 

(X-Xk) 

in two different ways and by comparing the coefficient of 
(1/x).^ By virtue of the remainder theorem the right-hand 
side of equation (8) may be written 

(x-Xk) 

. \Xk — -y-t-ti) . . . \Xtc Xn) . \Xk .yj-ij . . . \X]{ Xn) 



(8) 



= polynomial in x 



x^ 



■'^k 



+ ... . (9) 

On the other hand, 

{x-Xi)(x-x^) . . . (^-Xfc_i)(;r-^fc+i) . . . (x—Xn.) 



and 



= ^"-^ - kpW-' + fc/'2 ^"-' - • 



1 



(10) 



(x — x-i^){x — X2) . . . (x — Xi) 

111 1 



'0-f)0-?) 0-f) 



aPor this elegant proof of identities (7) and (7') the authors are 
indebted to D. Shanks of the Naval Ordnance Laboratory. 






'' ^i 1 _^i+i ^1 "T ^i+2 ^2 + 



(11) 



The coefficients of l/x obtained by multiplying (10) by 
(11) are, precisely the left-hand side of identities (7) and 
(7'). The right-hand sides of equations (7) and (7') are 
given by the coefficients of l/x in (9). If ^ = 1, 2, ... , i 
we obtain the expression given in (7) . On the other hand, if 
k = i-\-l,i-{-2, ...,n then the coefficients of l/x in (9) 
are zeros. 

Finally, we must justify the validity of the recursion re- 
lationship for the q functions given in equation (3). This 
can be accomplished by a simple argument based on the 
definition of these functions. ql~^ is the sum of all distinct 
products on the letters x^, x^, . . . , Xt-x raised to exponents 

(Xi , (Xi , . . . , (Xi such that %a.i = s. q\ is a similar sum 

1 2 *-i 

involving in addition the letter xu 

In order to obtain ql it is obvious that we must include in 
our summation all products contained in ^g"^. These do not 
involve the letter Xt. In addition, we must insert all products 
such that %(Xi = s which do contain Xt. It is apparent that 
all terms Xt ql-t possess this property. Also, it is seen easily 
that all terms involving Xt, and such that 2ai = s, are ob- 
tained in this manner. For, assume x^i x^2 . . . x^t is a term 
such that ^^i — s, fit^ 1, is not contained in Xt ql.i. Then 
x^i x^2 . . . x^t-1 is not contained in qg.^. This is contrary to 
the definition of g|_i which must contain all terms such that 
2 exponents = s — I. 
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CERTAIN PROBLEMS leading to complicated par- 
tial or integro-differential equations have recently been 
approached and some actually solved by utilizing various 
probability techniques and sampling methods. Collectively, 
these methods have become known as the Monte Carlo 
Method. 

The problems to which Monte Carlo techniques have 
been applied seem to be divided into two types. Typical of 
the first type is the problem of neutrons diffusing in mate- 
rial media in which the particles are subjected not only to 
certain deterministic influences but to random influences as 
well. In such a problem, the Monte Carlo approach consists 
in permitting a "particle" to play a game of chance, the 
rules of the game being such that the actual deterministic 
and random features of the physical process are, step by 
step, exactly imitated by the game. By considering very 
large numbers of particles, one can answer such questions 
as the distribution of the particles at the end of a certain 
period of time, the number of particles to escape through a 
shield of specified tliickness, etc. One important charac- 
teristic of the preceding approach is that the functional 
equation describing the diffusion process is by-passed com- 
pletely, the probability model used being derived from the 
process itself. 

A more sophisticated application of Monte Carlo Methods 
is to the problem of finding a probability model or game 
whose solution is related to the solution of a partial differen- 
tial equation, or, as in the present paper, to determine the 
least eigenvalue of a differential operator by means of a 
sampling process. As an example of how the latter problem 
might be attacked, we quote from a paper of Metropolis 
and Ulam •?■ 

"For example, as suggested by Fermi, the time independ- 
ent Schrodinger equation 

/\<^{x,y,z~) = (\— V)(l>ix,y,2) 



*This paper (except for the two appendices) was written while the 
authors were associated with the National Bureau of Standards 
at the Institute for Numerical Analysis. It appears in the Journal 
of Research of the National Bureau of Standards under the title 
"A Sampling Method for Determining the Lowest Eigenvalue and 
the Principal Eigenfunction of Schrodinger's Equation." The 
preparation of the paper was sponsored (in part) by the Office of 
Naval Research. 



could be studied as follows. Reintroduce time dependence 
by considering 

u(x, y, s, t) = <t>(x, y, 3) e-^'* ; 
then, u will obey the equation 
du 



dt 



= Am — Vu 



This last equation can be interpreted, however, as describ- 
ing the behavior of a system of particles each of which per- 
forms a random walk, i.e., diffuses isotropically and at the 
same time is subject to multiplication, which is determined 
by the value of the point function F. If the solution of the 
latter equation corresponds to a spatial mode multiplying 
exponentially in time, the examination of the spatial part 
will give the desired <f>{x, y, z) — corresponding to the 
lowest 'eigenvalue' A."* 

The main purpose of the present paper is to present an 
alternative method for finding the lowest eigenvalue and 
corresponding eigenfunction of Schrodinger's equation. The 
chief difference between the two approaches is that ours 
involves only a random walk eliminating entirely the multi- 
plicative process. This alteration in the model seems to sim- 
plify the numerical aspects of the problem, especially if 
punched card equipment is to be used. Apart from the pos- 
sible numerical simplification, the method is based on a 
mathematical theory which in itself is of some interest. 

The Mathematical Theory 

LyCt x^, X2, x^, . . . be independent identically distributed 
random variables each having mean and standard devia- 
tion 1 and let sjc = x^^ -\- X2 -\- ... -{■ x^. Under certain 
general assumptions on V{x), the most severe of which is 
that V{x) be non-negative, it can be shown^ that the limit- 
ing distribution function o-(«,0 of the random variable 



k^nt \ 

/ / e-''-'* da(Tia,t)dt = I 



(1) 



is such that 



xj/{x)dx , (2) 



aTo the best of our knowledge, this method has not been tried out 
numerically. 
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where \l/{x) is the fundamental solution of the differential 
equation 



l2^-[s+V(.)],.0 



(3) 



subject to the conditions 

\1},'(X)\<M Xy^O 

'A'(+0)-V'(-0) =2. 
The fundamental solution ipix) of (3) is expressible in 
terms of the normalized eigenf unction '{i/'i(^) }■ and eigen- 
values A; of the one-dimensional Schrodinger eigenvalue 
problem'' 

1 d-xl/ 



as 



(5) 



Thus, from (5) and (2) 



ilj{x)dx = I I e-'^-'^d, 

00 «/0 t/o 



aa{0t,,t)dt 



=E 



/^oo 

«/ 00 



(6) 



:y-f Ay 
Inverting (6) with respect to s there results 

/oo /^oo 

j •/ — 00 

and therefore the following expression for Aj is obtained, 

/oo 
e~°- da(T{(x,t) . (8) 

If in (7) all terms in the expansion but the first are 
neglected. 



^ log xp 




/'oo 
x(0) / ^, 

•y — 00 



(x)dx — Ai^ , 



/oo 



(9) 



Thus, if, by choosing a finite t, an attempt is made to cal- 
culate Ai from (8), there are two sources of error. The first, 
usually a small source of error, is from the exponentials 
neglected in the expansion (7). The second, and more im- 
portant, is from neglecting the term- log< \p-^{0)j \p-i^{x)dx\. 



I'From here on all the steps are formal. In all cases of physical 
interest they can be justified rigorously. 



This latter source of error is especially significant since, as 
will be apparent shortly, it is impractical from other points 
of view to take t very large. All of this difficulty may be 
obviated by considering (7) for two distinct values of t, 
say t^ and t^ ; then, if the exponentials after the first are 
neglected as before on dividing there results, 



K 



I. 



e-" da(T{(x,t^) 



/ e-" da(T{ci,t^) 



(10) 



The Monte Carlo process consists in the calculation a(a,fi) 
and o-(a,^2) by a sampling process. 

If, instead of a{o!,,t), the limiting distribution (T^{a,t) of 
the random variable is considered. 



^EK^+v 



k<nt 






(11) 



then o-|(a,0 also satisfies (2) and (3), but now the condi- 
tion i/'X+O) - lA'C-O) = 2, is replaced by ^''(1+) - 
il/'{$—) = 2. Therefore, repeating steps (4) -(7) 



/oo /'oo 

^-«rfa(r|(a,0 =y^^~V<Ai(l) / il'jix)dx 
i %J — 00 

/oo 
mn — 

<— >00 / 

/ e-''daCr{(X,t 



Ml) 



(12) 



(13) 



so that the principal eigenfunction also can be calculated. 
The extension of the preceding method to multidimen- 
sional Schrodinger equations is immediate. It is in these 
cases that the method will probably prove to be most use- 
ful since, unlike the standard variational procedures, the 
extension to several dimensions seems to cause compara- 
tively little difficulty. For illustrative purposes, consider 
Schrodinger 's equation in three dimensions. Here three 
independent sequences 

Xi, X2, Xg, . . . 
Y Y Y 

■^1> ^2> '^3» • • • • 

of independent identically distributed random variables, 
each having mean and standard deviation 1, must be 
considered. Let .y^j.^ = X^ -|- Xg + . . . + X^, and Syy,, Sgu 
have the obvious meanings. Consider the limiting distribu- 
tions (j{ci,t) and (t(|, 1J, ^; of the random variables 

fn) 



and 



»2-^ V 



V« V 



<. I ^xk I ^yk 5, 1 Szk 



)■ 



respectively. 
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In exactly the same way as in the one-dimensional case 



Ax 



and 



V^, (0,0,0) t^^ 



/oo 


/oo 



dofT{ty],V, 



/ 



^-« da(r(<X,t) 



So far, the theory was carried out under the assumption 

that the potential function V was non-negative. In most 

cases of physical interest this is not so. For the hydrogen 

atom, for instance 

r.. s const. 

V{x, y,s) = — ■ . 

V^^ -\- y^ -\- 2^ 
However, the modification is easy, although, as yet, all the 
points of mathematical rigor have not been clarified. 
The formula for the lowest eigenvalue now becomes 



Ai 



r 

'2 *1 / 



/ 

%/ — 00 



e-°- daa{(l,t^) 



and a corresponding modification needs to be made in the 
formula for the principal eigenf unction. We have not yet 
tested numerically any case with a negative potential func- 
tion, but we hope to be able to report in the near future. 

Numerical Examples and Discussion 

The Monte Carlo procedure used here consists in the x;al- 
culation of the distribution function cr{(x,t) by a sampling 
process; the principal eigenvalue is then calculated from 
(10). For the purposes of numerical illustration consider 
two examples, V{x) = x^ and V{x) = \x\. In both of 
these cases the eigenvalues are known; hence there is a 
check on the accuracy of our procedure. In order to calcu- 
late (r{(X,t), say when V{x) = x^, it is seen from (1) that 
the limiting distribution must be considered as w — > <x of 



fc<n* 



•Jfc 



(14) 



This means from our point of view that we must (a) 
choose a distribution for the X's; (b) choose a sufficiently 
large n; (c) select an appropriate ^ ; (d) calculate for nt X's 
the normalized sum (14) ; (e) repeat (d) many times so 
that the empirical distribution may be obtained from these 
many samples. 



Although, under the conditions mentioned previously, 
the distribution function cr(<x,t) is independent of the dis- 
tribution of the X's, the actual numerical calculation of 
a{(x,t) is expedited by choosing the distribution of the X's 
to be the Bernoulli distribution, i.e., 

P(X= 1) =P(X = -1) =i . 
The sequence of random variables X^, Xj, Xg, ... is then a 
sequence of -j- I's and — I's such as might be obtained in 
coin tossing. This is conveniently and rapidly achieved on a 
calculating machine by considering sequences of random 
digits, counting even digits -f- 1 and odd digits — 1. 

The value of n to be used must be large enough so that 
the empirical distribution function calculated is close to the 
theoretical limiting distribution function (T(<x,t). From (10) 
it is seen that the two values of t, t^^ and ^2, to be used in the 
calculation of A, must be large enough so that the exponen- 
tial terms neglected are sufficiently small. However, since 
the sample size is nt, the desire to make both n and t large 
must be tempered by practical considerations. The number 
of samples to be used must be large enough so that the 
empirical distribution adequately represents (T{(x,t). Before 
discussion of these points in more detail, let us consider 
an actual numerical computation. The following data for 
V{x) = x^ and V{x) = \x\ were calculated from a certain 
set of random digits'^ on the IBM Type 604 Electronic Cal- 
culating Punch. For both x^ and \x\ n was selected to be 
400, ^1 = 5, ^2 = 3.75, and 100 samples were used. In col- 
umn A^ of Table I is tabulated 



8000 



E 



Sjc\ 



^2 = 



B,= 



_1 
8000 

1_ 

160000 



fc=i 

1500 

E 



•ffc 



S]c' 



B. = 



1 



160000 






Since each RAND random number card contains 50 ran- 
dom digits and 2000 digits are needed to form one sample, 
a set of 4000 RAND cards was sufficient for this experi- 
ment. It takes 20 minutes on the 604 to calculate A^ and A2 
for one sample (similarly for B^ and ^2) so that it takes 
approximately 35 hours to secure the following data. 
In order to calculate Ai from (10) the values of 



e-"' daai<x,tj and / ^"^ 

«/oo 



da(r(<X,t2) 



are needed. Both of these integrals were calculated numeri- 
cally from the data in Table I by adding the exponen- 
tials of the entries in appropriate columns. For the case 
V{x) ^ I ;ir|, the values ofthe integrals were obtained from 

100 100 

— -Y^ e-^i, and 77^ T^ e-^2i and for F(^) = x^ from 
100-^—/ 100 Z-/ 



100 



100 

E 



and 



100 

1 
100 



E 



In the case V^x) = \x\ 



cThis set of random digits was prepared by the RAND Corpora- 
tion, Santa Monica, California. 
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the true lowest eigenvalue to two places is .81 and in the 
case V{x) = .rMt is (V272) = .71 . 

first 50 second 50 all 100 

samples samples sainples 



Tablk I {Continued) 
Ax A2 Bx 



B, 



1. 

2. 

3. 

4. 

5. 

6. 

7. 

8. 

9. 
10. 
11. 
12. 
13. 
14. 
IS. 
16. 
17. 
18. 
19. 
20. 
21. 
22. 
23. 
24. 
25. 
26. 
27. 
28. 
29. 
30. 
31. 
32. 
ZZ. 
34. 
35. 
36. 
37. 
38. 
39. 
40. 
41. 
42. 
43. 
44. 
45. 
46. 
47. 
48. 
49. 
50. 
51. 
52. 
53. 
54. 
55. 
56. 
57. 
58. 
59. 
60. 



.83 
.80 



.79 
.69 



Ax 



TABI.E; I 

A2 



Bx 



1.332 
3.808 
2.795 
8.723 
4.169 
1.195 
6.674 
4.103 
5.751 
4.250 
2.909 
2.834 
1.888 
2.022 
1.680 
7.700 
3.228 
1.844 
2.376 
4.533 
4.209 
3.847 
5.206 
1.962 
3.523 
3.605 
9.168 
5.625 
3.215 
3.293 
2.710 
3.638 
7.928 
3.475 
1.958 
5.356 
3.297 
1.883 
4.849 
2.282 
4.253 
7.559 
1.122 
1.433 
2.984 
1.677 
5.145 
4.982 
1.675 
1.580 
2.606 
4.303 
7.553 
3.138 
3.343 
1.423 
7.996 
1.916 
1.490 
3.536 



2.101 

5.707 
5.545 

13.917 
4.766 
2.136 

12.112 
5.242 
8.840 
4.981 
5.643 
3.416 
2.194 
2.337 
3.908 

12.712 
4.973 
2.654 
5.275 
5.640 
7.543 
5.590 
6.314 
2.698 
4.807 
5.397 

13.692 
7.898 
4.152 
3.972 
3.683 
7.386 

11.567 
3.846 
2.594 
7.216 
6.395 
2.682 

10.020 
3.067 
6.957 
8.834 
2.587 
3.602 
3.477 
3.158 
9.697 
9.140 
2.152 
3.375 
3.621 
5.883 

12.092 
5.703 
4.327 
2.315 

11.458 
2.744 
2.161 
5.382 



.738 
5.295 
3.460 

30.722 

6.123 

.598 

15.258 
5.826 

10.440 
6.069 
3.844 
3.602 
1.595 
1.510 
1.184 

24.769 
3.563 
1.523 
2.017 
6.380 
7.735 
6.666 
9.148 
1.859 
5.084 
5.014 

27.973 

11.528 
3.362 
4.136 
3.006 
6.114 

25.084 
4.396 
1.457 
8.888 
3.800 
1.387 
8.396 
2.258 
6.744 

17.143 

.500 

.869 

2.851 

1.025 

11.941 
9.201 
1.129 
1.147 
2.254 
7.987 

19.632 

3.968 

4.264 

.770 

21.405 

1.253 

.789 

4.125 



.81 
.75 



B, 



1.290 
8.502 
9.610 

52.616 
6.602 
1.453 

39.058 
7.004 

18.143 
6.679 

10.044 
4.097 
1.700 
1.638 
5.289 

44.936 
6.145 
2.183 
8.854 
7.616 

16.703 
9.204 

10.600 
2.923 
6.482 
7.680 

44.461 

16.010 
4.170 
4.677 
4.025 

17.413 

36.001 
4.550 
1.948 

11.781 

11.555 
1.995 

30.233 
2.882 

13.502 

18.702 
2.634 
4.855 
3.161 
3.341 

28.826 

23.092 
1.472 
3.807 
3.247 

10.104 

36.427 
9.325 
5.380 
1.533 

31.585 
1.959 
1.404 
7.461 



61. 
62. 
63. 
64. 
65. 
66. 
67. 
68. 
69. 
70. 
71. 
72. 
72>. 
74. 
75. 
76. 
77. 
78. 
79. 
80. 
81. 
82. 
83. 
84. 
85. 
86. 
87. 
88. 
89. 
90. 
91. 
92. 
93. 
94. 
95. 
96. 
97. 
98. 
99. 
100. 



6.278 
1.174 
5.272 
3.576 
2.686 
6.166 
4.412 
1.732 
2.730 
1.331 
3.668 
2.948 
5.279 
11.284 
2.183 
4.298 
1.412 
2.002 
2.619 
2.695 
4.706 
5.517 
1.755 
2.084 
1.574 
5.818 
4.387 
3.666 
7.111 
6.280 
4.872 
3.035 
3.608 
3.977 
3.614 
9.193 
4.108 
4.355 
1.898 
9.683 



11.060 
1.709 
9.919 
5.277 
5.081 
8.785 
5.548 
4.664 
4.970 
2.772 
5.318 
3.801 
8.931 

15.784 
2.787 
7.427 
2.893 
2.567 
3.780 
4.616 
6.672 
7.981 
2.099 
2.876 
2.743 

10.444 
7.844 
5.244 

13.311 
8.548 
5.836 
3.323 
4.697 
4.964 
7.867 

11.183 
5.282 
5.744 
2.661 

17.154 



16.187 
.573 

10.463 
6.021 
3.577 

14.056 
6.095 
1.170 
2.440 
.765 
5.730 
3.248 

12.455 

41.684 
2.086 
9.139 
.894 
1.602 
2.981 
2.974 
8.330 
9.846 
1.066 
1.735 
.993 

13.322 
9.037 
4.416 

19.834 

13.403 
9.230 
2.865 
4.256 
5.215 
5.757 

27.789 
5.918 
7.774 
1.507 

35.803 



34.779 

.919 

27.825 

8.707 

8.307 

19.627 

7.182 

8.720 

6.639 

2.642 

8.001 

4.104 

23.513 

58.525 

2.455 

17.157 

3.490 

2.024 

4.205 

6.077 

11.487 

14.805 

1.207 

2.504 

2.256 

30.522 

18.695 

6.541 

50.822 

17.587 

10.023 

2.973 

5.357 

6.242 

20.434 

31.075 

7.374 

10.223 

2.013 

80.543 



An interesting feature of the data is that column B^, for 
example, may be thought of as arising from n = 2000, 
t = 1 ; w = 1000, t = 2, etc., as well as from n = 400, 
^ = 5. The larger n is taken the closer will be the empirical 



distribution to <7(a,0- The value of 



/oo 
e'*^ da(r{<x,t) in the 



case t = \ and V{x) = x^ can be calculated exactly and, 
to three places, is .678. Calculating this integral from col- 
umn B2 with n = 2000, ^ = 1 there results .685. 

Instead of using Bernoulli distributed variables, one 
might use other distributions. One definite advantage of 
Bernoulli distributed variables is that the computation util- 
izes only the crudest properties of the random digits, i.e., 
whether they are even or odd. One possible advantage for 
certain other distributions is that n might not have to be 
taken so large. In particular this should be true if Gaussian 
distributed variables are used. RAND Gaussian deviates 
were used in constructing Table II. Here t^ = 3.75, ^2 = ^ 
were chosen as before, but now m = 100. This means sam- 
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pies of size 500 instead of 2000 and, therefore, a total 
machine computation time of 9 hours for 100 samples. 

Envisaging the possibility of calculating the second eigen- 
value, we considered the quantities 

375 500 

10-*2(^fc + 5)^andl(>-^^(^fc + 5) , (15) 

which correspond to taking ^ = 0.5 in (11). This should 
not make any difference in the calculation of the lowest 
eigenvalue ; consequently, Table II can be utilized in the 
same way as Table I. It should be borne in mind, however, 
that columns C-^ and Cg of Table II represent experimental 
values of the quantities (15) with the .s^'s being sums of 
Gaussian deviates. Asterisks on the entries of Table II in- 
dicate that J375 or .fgoo was negative. Although this informa- 
tion is unnecessary for the purpose of calculations of the 
lowest eigenvalue, it is used in the calculation of the next 
eigenvalue. How this can be done is explained briefly in 
section 3. 

TabIvE II 



Cx 



C2 



Ci 



C2 



1. 


3.963 


13.997 


2. 


2.348* 


2.721 


3. 


1.889* 


2.231 


4. 


3.817* 


4.372* 


5. 


2.395 


2.802* 


6. 


12.467 


27.683 


7. 


1.068* 


1.115* 


8. 


.578* 


.967* 


9. 


1.504* 


1.760 


10. 


4.249 


9.787 


11. 


1.751* 


3.316* 


12. 


9.922 


13.975 


13. 


5.680 


6.649 


14. 


3.348 


4.684 


15. 


13.431 


24.769 


16. 


1.473* 


2.615* 


17. 


45.262 


74.304 


18. 


1.157* 


3.109 


19. 


.906* 


3.644* 


20. 


4.601 


12.641 


21. 


8.356* 


13.084* 


22. 


.308* 


1.874* 


23. 


2.111* 


4.428* 


24. 


2.035* 


2.726 


25. 


1.625* 


3.898 


26. - 


1.879* 


5.713 


27. 


1.333 


8.961 


28. 


1.769 


2.319* 


29. 


1.906* 


2.417* 


30. 


7.522 


7.979* 


31. 


14.934 


15.493* 


32. 


4.282* 


13.747* 


33. 


1.356* 


2.718 


34. 


4.448 


8.329 


35. 


1.306* 


2.112* 


36. 


5.465* 


12.396* 


37. 


2.918 


9.006 


38. 


35.821 


56.626 


39. 


9.814* 


11.707* 


40. 


1.811 


4.628 


41. 


3.374 


6.460 


42. 


1.633 


6.578 


43. 


6.976 


8.705 



44. 


7.595 


13.096 


45. 


2.336 


2.862 


46. 


1.931* 


2.579* 


47. 


4.507* 


8.294* 


48. 


8.581* 


11.301* 


49. 


4.698* 


8.220* 


50. 


10.154 


16.884 


51. 


8.828 


15.387 


52. 


.997* 


1.604* 


53. 


8.582 


12.838 


54. 


2.730* 


3.249* 


55. 


10.471 


13.140 


56. 


1.456* 


1.845* 


57. 


8.230* 


11.924* 


58. 


5.515* 


8.146* 


59. 


2.202* 


3.182* 


60. 


2.129* 


9.650* 


61. 


49.929 


84.307 


62. 


3.095 


6.220 


63. 


2.585* 


3.867* 


64. 


1.318* 


1.906* 


65. 


2.241* 


9.102* 


66. 


9.737 


12.185 


67. 


9.738* 


11.347* 


68. 


4.604 


5.589 


69. 


20.364 


29.707 


70. 


1.097* 


1.895* 


71. 


5.891 


13.391 


72. 


9.019* 


12.815* 


73. 


1.843 


6.280 


74. 


7.961 


11.431 


75. 


1.109 


1.926 


76. 


2.417* 


3.037 


77. 


2.402* 


2.797* 


78. 


2.043* 


4.214* 


79. 


.582* 


.752 


80. 


2.429 


4.381 


81. 


8.584 


13.034 


82. 


2.793* 


4.659* 


83. 


1.557 


5.023 


84. 


1.006* 


1.677* 


85. 


5.327* 


15.531* 


86. 


2.058 


7.307 







Tabi^e; II {Continued) 








Cx 


C2 




Cx 


C2 


87. 


12.481 


12.803* 


144. 


1.421* 


3.652* 


88. 


.925* 


2.403 


145. 


2.669* 


3.889* 


89. 


15.519 


19.879 


146. 


1.125* 


2.071 


90. 


1.456* 


4.009* 


147. 


3.439* 


5.824* 


91. 


5.244 


10.047 


148. 


1.269* 


5.651 


92. 
93. 


2.497 
16.700 


2.931 
17.382 


149. 


7.949 


14.252 


94. 


4.471* 


8.156* 


150. 


2.808* 


4.434* 


95. 


2.139* 


2.580* 


151. 


6.413 


12.573 


96. 


1.196* 


2.273* 


152. 


5.936* 


10.432* 


97. 


23.014 


28.420 


153. 


9.788 


18.093 


98. 


6.959* 


7.022* 


154. 


1.251 


2.707 


99. 


3.374* 


8.651* 


155. 


34.799 


63.494 


100. 


1.600* 


2.250* 


156. 


.841* 


1.742* 


101. 


4.626* 


7.794* 


157. 


1.461* 


1.615* 


102. 


5.195* 


10.215* 


158. 


7.220 


21.125 


103. 


6.837* 


12.160* 


159. 


1.275* 


2.359* 


104. 


1.913 


4.412* 


160. 


9.355 


19.565 


105. 


2.276* 


2.770 


161. 


3.546 


4.040* 


106. 


1.935* 


3.983* 


162. 


1.238* 


2.304* 


107. 


3.560 


5.808 


163. 


.975* 


1.283 


108. 


2.857* 


7.384* 


164. 


1.980 


3.659* 


109. 


3.399* 


4.484* 


165. 


14.620* 


17.565* 


110. 


6.917* 


27.107* 


166. 


2.378* 


2.865 


111. 


38.661 


56.479 


167. 


2.279 


6.279 


112. 


1.891 


2.000* 


168. 


10.846 


13.754 


113. 


8.506 


9.646* 


169. 


1.341* 


3.673* 


114. 


4.209 


10.461 


170. 


1.861* 


2.729* 


115. 


33.194 


44.247 


171. 


28.588* 


53.358* 


116. 


.869* 


1.289 


172. 


2.943 


3.087* 


117. 


3.946 


15.204 


173. 


1.694 


2.568 


118. 


1.680* 


2.995* 


174. 


1.729 


2.516* 


119. 


8.178* 


9.056 


175. 


.844* 


1.449* 


120. 


27.621 


51.650 


176. 


3.441* 


5.380* 


121. 


8.444* 


15.836* 


177. 


.712 


.838* 


122. 


13.254 


16.467 


178. 


1.222* 


3.469* 


123. 


1.755* 


5.246* 


179. 


20.852* 


42.263* 


124. 


36.902 


70.134 


180. 


7.575* 


16.552* 


125. 


7.567* 


18.553* 


181. 


2.042* 


3.058 


126. 


14.725 


38.551 


182. 


6.797 


7.343* 


127. 


.983* 


2.572 


183. 


1.536 


4.499 


128. 


24.529 


31.660 


184. 


1.235 


1.472* 


129. 


2.042* 


2.385* 


185. 


3.541* 


4.272 


130. 


3.587* 


9.858* 


186. 


5.394* 


6.149* 


131. 


1.335* 


1.755 


187. 


18.589 


28.787 


132. 


2.142* 


3.801* 


188. 


8.372 


12.628 


133. 


16.361 


27.759 


189. 


13.470* 


23.775* 


134. 


8.193 


9.301 


190. 


1.196* 


2.851* 


135. 


2.719* 


5.141* 


191. 


3.507 


7.712 


136. 


4.767* 


12.737* 


192. 


21.489 


43.372 


137. 


42.230 


108.038 


193. 
194. 


1.038* 
3.591* 


1.109* 
3.864 


138. 


41.236 


78.423 


195. 


2.295* 


2.529* 


139. 


8.243 


9.911* 


196. 


1.044* 


2.320 


140. 


31.522 


36.393 


197. 


4.865* 


6.759* 


141. 


2.600 


2.881* 


198. 


1.184 


11.852 


142. 


1.932 


2.055* 


199. 


1.228* 


1.969* 


143. 


8.562 


10.813 


200. 


9.346* 


22.204* 



Using the data in Table II and again ( 10) , we computed 
the following approximations to Ai (actual value = .71). 



first 50 
samples 



second 50 
samples 



third 50 
samples 



fourth 50 
samples 



all 200 
samples 



Ai 



.82 



.72 



.83 



.64 



.74 
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For both V{x') = \x\ and V(x) = x^ dll the eigenvalues 
are known, so that in the two illustrative examples above 
it was easy to choose appropriate values for t^ and ^2- The 
proper value for n and the appropriate number of samples 
were determined experimentally. In general, when all eigen- 
values are unknown, the following rule-of-thumb procedure 
seems to be feasible. Having first made a guess at an appro- 
priate n and t, and a certain number of samples, compute A^. 
Repeat the computation now for the same n, the same num- 
ber of samples and successively larger t's until the calcu- 
lated values of Ai become stable to the desired number of 
places. If they do not become stable, the number of samples 
must be increased. Keeping two values of t, for which the 
calculated A's had the stable value, increase n and see if 
the calculated value of A changes. If not, n is sufficiently 
large. If it does change, increase n until a new stability 
appears. This stable value is then the appropriate approxi- 
mation to Ai. The mere fact that stability is observed means 
the number of samples is sufficiently large. 

The Second Eigenvalue 

If the principal eigenfunction is even, then it is possible to 
extend the theory of section 1 in such a way that the calcu- 
lation of the second eigenvalue becomes feasible. Without 
going into any details, we just state the pertinent result: 
Let 



EKi) =limii^^ 

«— *oo I 

and 



-It/u+ir,) 



Vw 



£7(1) = lim£-^^ nk^nt 






Vn' 



Snt> 



Snt< 



(16) 



(17) 



the mathematical expectations on the right-hand sides being 
conditional expectations under the conditions Snt > and 
^nt < 0, respectively. 
Thus 



^-iEt{0-B^(^)[=J^e-h^ 



2 
where 



M^) i 
%j — 00 



(18) 
p{x) il>j(x)dx , 



,(x) = 



'+1,^>0 

^^^^=^ l,x<0. 
If t^ and ^2 are sufficiently large, 

1 . EU$)-E7(i) 



{- 



log 



(19) 



h-h ''Et^i$)-Ej^(i) 
From the discussion of section 2 it should be clear how one 
applies (19) when the data of Table II are available. One 
must see to it that $ is so chosen that if/zii) =7^ 0; otherwise 
a higher eigenvalue may have been calculated. From the 
data of Table II the following is obtained 

^2-^1.1 , 
whereas the exact value is \/2 = 1.41. The poor agreement 
could have been expected in view of low accuracy in the 
calculation of A^. 



In conclusion, we wish to thank Dr. E. C. Yowell of the 
National Bureau of Standards for wiring the control panels 
and for excellent supervision of all the punched card work. 

Appendix I 

We give here an intuitive approach to the mathematical 
theory of section 1. This approach was suggested to us by 
Dr. G. A. Hunt. 

Consider the simple random walk in steps 

± A (A = 1/V^) 
each step being of duration t. At each stage the particle has 

the probability t Visjc A) = (1/w) V{su/'\/n) of being 
destroyed, where j^ A is the displacement of the particle 
after time k t (k steps). 

In the limit as «-> 00 ( A-^0, t— »0, A^/t = 1 ) we are led 
to a continuous diffusion process with destruction of matter 
governed by the function V(x) ^0. 

The probability Q(x,t)dx that the particle will be found 
between x and x -\- dxsit time t can be found by calculating 
the Green's function (fundamental solution) of the differ- 
ential equation 

Bt 2 dx^ '^^ ''^ ' 
i.e., that solution of the equation which for f->0 satisfies 

Q{x,t)^h{x) . 
The integral 



/ 



Q{x,t)dx 



represents the probability that the particle will survive dur- 
ing the time interval (0,0- 

In terms of eigenvalues and normalized eigenfunctions of 
the Schrodinger's equation (4) we can express Q(x,t) as 
follows : 

0(^,0 =^e-h'xj{0)xj{x) . 

It is, of course, understood that V{x) is assumed to yield a 
discrete spectrum. In the case when continuous spectrum 4s 
also present the formula has to be modified but the calcu- 
lations of the lowest eigenvalue are, in general, not affected. 
Finally, 



/ 



Q(x,t)dx =£^e-^s*xp^ 



E< 



•^00 

%J — 00 



)dx , 



and it remains to verify that 



/ Q(x,t)dx= I 



e-°- da(T(<x •,t) 



First note that the expectation (average) of 



80 
approaches 



/ 



e-°'daa{(x;t) 



as«- 



00 (because the distribution function of-N ^ Vl — ^ ) 

approaches ar((x;t) and V{.r) ^ 0). 

On the other hand, using the approximation 



tV(su/S) 



I-tV(s^A) 



note that 



,-. 2 



lt<nt 



FUA) 



is approximately the probability of survival of the particle 
if its consecutive displacements are JiA, S2/S., • • • , SntA. 
In taking the expectation of 

we average the probability of survival over all possible 
choices of successive displacements (all possible paths) and 
thus obtain the unconditional probability of survival. This 
unconditional probability of survival approaches, as n—> 00 , 
the integral 






Q{x,t)dx . 
On the other hand, it also approaches 

e-<'da(T(<x;t) 
and consequently 

/oo /^oo 

e-"'da(T{(x;t) = / Q(x,t)dx 



= ^.-^/^,(0) / 



il/j{x)dx . 



Although it has been assumed V{x) ^ 0, all considerations 
are applicable to potentials which are bounded from below. 
Although atomic and molecular potentials become nega- 
tively infinite, they can be cut off sufficiently low without 
changing appreciably the eigenvalues and the eigenfunc- 
tions. 

Appendix II 
In this appendix we sketch a Monte Carlo Method for find- 
ing the lowest frequency of a vibrating membrane. In 
mathematical terms, find the lowest eigenvalue of the equa- 

-xAu -\- Xu = , 

valid in a region Q, and subject to the boundary condition 

u = , 
on the boundary T of O. (This corresponds to the case of 
the clamped membrane.) The lowest frequency is the square 
root of twice the lowest eigenvalue. 



C O M P U T A T I O N 

Cover the region Q with a square net, the side of the square 
being A. Starting from a point (x^, y^) = (wA, wA) in- 
side O, consider a two-dimensional random walk in which 
a point is allowed to move to each of its four nearest neigh- 
bors, the choices being equiprobable (probability 1/4). The 
duration of each step is t related to A by the equation 

^ = 1. 

T 

Consider the boundary r of O as an absorbing barrier. This 
means that whenever the particle, performing the random 
walk, leaves the region O it is destroyed. 

In the limit as A-»0, the probability Q{xo, yo', t) that 
the particle will survive during the time interval (0,0 can 
be obtained as follows : 

Q{^o,yo;t) = J J Pi^o,yo\^,y;t) dxdy , 

where P (.*■(,, y^ \ x, y; t) is the fundamental solution of the 

dP 1 
differential equation -^r— = ;r A-P , 
ot Z 

subject to the boundary condition 

P = Oonr, 

and the initial condition 

as ^->0. 

This fundamental solution can be expressed in terms of 
the eigenvalues and normalized eigenfunctions of the mem- 
brane problem as follows : 

P{x„ y^\x,y;t) = }e-^i* ij^jixo, yo) ^j(^, y) • 



Thus 



Q i^o, yo ; = 2 ^~^'' '^^■'^■*'*" '^"U J '^'^^' ^^^^ ^^ ' 



o 



and 



t—yco t 

The probabiHty Q(xo, y^; t) can be calculated by a sam- 
pling method in the following way. Start A^o independent 
particles from (xq, y^) and let each of them perform the 
random walk described above. Watch each particle and 
count the number Nt of those particles which have not left 
the region during the time interval (0,0- 
Then 

Nt 



and 



Q{^o,yo',t) 
1 



Ai 



No 
Nt 



t^^'^N 



The practicality of this method has not been tested. 
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DISCUSSION 

Mr. Bisch: I was very much interested in the clear dis- 
cussion and the problem you chose, which is part of a prob- 
lem we have to solve. My first question concerns that func- 
tion, V(x) or V(x, y, z). Could you have that function 
determined experimentally — in other words, not expressed 
algebraically ? 

Professor Kac: Yes, 

Mr. Bisch: The second question is about the boundary: 
Could you in this method leave the boundary as a simple 
unknown temporarily ? 

Professor Kac: To which problem are you referring? Do 
you mean here there is no boundary ? 

Mr. Bisch: There is no boundary in a problem of the 
membrane. You made the boundary zero. In other words, 
your deflection was zero. Could you leave that deflection 
temporarily unknown as a quantity like £/„ ? 

Professor Kac: I think so. Actually, all I can say with 
certainty is the following : If you have the other boundary 
condition, (du/dn) = 0, the other classical condition, then 
all you have to do is to have the boundary not absorbing but 
reflecting. Now, the mixed boundary au -{- b (du/dn) = 
can again be done in principle by making the boundary 
partially reflecting, partially absorbing. When you come to 
the boundary you play an auxiliary game, which would de- 
cide whether you are going to throw the particle out or keep 
it in the game. You see, this is only an eigenvalue problem. 
Consequently, you cannot just say that the solution must be 
f{x) on the boundary, because that would not give a char- 
acteristic value problem, and this is designed primarily to 
find eigenvalues. On the other hand, if it comes to Laplace's 
equation with a prescribed condition, then Dr. Yowell will 
speak about a random walk method which will do that. In 
fact, they have some experiments in the case of Laplace's 
equation in a square. 

Dr. King: One should emphasize the point of view of 
accuracy. I don't believe there is any hope of getting, say, 
six significant figures out of a Monte Carlo Method. 

Professor Kac: Agreed. 

Dr. King: But I disagree a bit with your point of view 
that it is worth doing, even in the very simplest cases, if 
you are not interested in accuracy. I think for the same 
amount of work you could work the harmonic oscillator 
with other methods and get one significant figure or two. 

Professor Kac: If I understood you correctly, you are 
saying that apart from the practical applications it is inter- 
esting because of the principle involved. Is that correct? 

Dr. King: Yes. 

Professor Kac: There I definitely agree. 

Dr. King: I think it is still practical, though, apart from 
being an amusing experimentation ; it is a practical method 
if you are interested in an engineering problem; so you 
only need a couple of figures. 



Professor Kac: With that I agree. As a matter of fact, it 
has another advantage, actually, which bore some fruit, not 
particularly exciting, but this way of looking at it produces 
even results of some theoretical interest. 

For instance, I am able — although I won't do it here be- 
cause it will be a bit too technical and probably too tedious 
— to justify the so-called W K B method, at least one as- 
pect of it, by diffusion analogies ; and there are other new 
viewpoints. If you can look at something from different 
points of view, it is certainly helpful, and often practical. 

On the other hand, for someone who has the tough atti- 
tude ("You give me the lithium molecule to ten per cent, 
or I won't take the method"), of course, one would still 
have to see what one can do, and, actually, I agree with you. 
What I am trying to do is to bend backwards in being 
cautious about it. 

Professor Tukey: It seems to me that there is a point to 
be made that came out in the discussion at the last confer- 
ence. That is, that one point of view for the use of Monte 
Carlo in the problem is to quit using Monte Carlo after a 
while. That, I think, was the conclusion that people came 
to then. That was the natural evolution and perhaps the 
desirable thing. After you play Monte Carlo a while, you 
find out what really goes on in the problem, and then you 
don't play Monte Carlo on that problem any more. 

I think the thing to suggest here is that, by the time peo- 
ple have played Monte Carlo on the lithium atohi, perhaps, 
or the lithium molecule, or something more complicated, 
people will get to the place where they won't be playing this 
simple Monte Carlo any more ; they will be playing Monte 
Carlo in some peculiar space where you have obtained ap- 
proximations to the wave functions as your coordinates, 
and not x, y, and s ; and then you will start to get more for 
a given amount of machine time. 

This is going to get arbitrarily complicated. When you 
start to cross-breed Monte Carlo with a standard method — 
and that isn't too far away in the handling of hard prob- 
lems — ^you are going to have to do something like that. 

Professor Kac: There is confirmation of that because of 
rather extensive calculations performed with relatively sim- 
ple equipment built for that purpose at Cornell, by Dr. 
Wilson and collaborators, in connection with his study of 
cosmic ray showers. They found the Monte Carlo Method 
most valuable because it showed them what goes on. I mean 
the accuracy was relatively unimportant. The five per cent 
or the seven per cent accuracy they obtained could be con- 
sidered low ; but all of a sudden they got a certain analytic 
picture from which various guesses could be formulated, 
some of them of a purely analytical nature, which later on 
turned out to verify very well. As a matter of fact, that is 
certainly one aspect of Monte Carlo that should be kept in 
mind. I agree that one of the purposes of Monte Carlo is to 
get some idea of what is going on, and then use bigger and 
better things. 



A Punched Card Application of the JS/Lonte Carlo JS/Lethod 
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IN THIS PAPER a description will be given of a 
punched card technique for applying the Monte Carlo 
Method to certain neutron problems. It should be kept in 
mind that this procedure is designed for a machine in- 
stallation consisting of a standard type 602 or 604 cal- 
culating punch, a collator, a reproducer, a sorter, and an 
accounting machine. Other combinations of machines would 
require a different approach, and an installation with a 
card-programmed electronic calculator would use an en- 
tirely different technique from the one about to be described. 

In any event, the problem may be stated as follows : 

Assuming a monochromatic point source of neutrons of 
given energy within an infinite medium of known constit- 
uents, hypothetical case histories for a number of these 
neutrons will be built up as they undergo a series of ran- 
dom collisions with the constituents of the selected medium. 
These collisions result in either an absorption or an elastic 
scattering of the neutrons, and the main work of this prob- 
lem is to follow the selected neutrons through successive 
collisions until they either are absorbed or fall below a cer- 
tain energy level. 

For each collision of each neutron the following will be 
recorded : the type of collision undergone ; the energy loss 
per collision, Am; the total energy loss at the end of the 
current collision, u; the ^-component of the distance trav- 
eled, p, between collisions, A^; the ^r-component of the total 
distance traveled, z; the angle of deflection after the colli- 
sion, w; and the direction cosine with the .s-axis, /*. 

When these data have been recorded, they may be used 
for various statistical studies to determine such things as, 
how far neutrons of a given energy may be expected to 
penetrate a certain substance. An example of how these 
data may be used will be given subsequently. 

To begin any such series of calculations, an arbitrary 
number of neutrons — say 1,000 numbered from 000 to 999 
— are selected and permitted, on punched cards, to go off 
in a random direction, travel a random distance, hit a type 
of particle determined in a random fashion, and lose an 
amount of energy determined also in a random way. For 
each neutron, a collision card is made bearing the neutron 
number, the collision number, and all the data (distance, 
direction, etc. ) pertaining to that particular collision. Then 



the neutrons are again permitted to go off in a random di- 
rection, travel a random distance, hit some type of particle, 
and lose energy. These data are recorded on a second set of 
collision cards, one for each neutron; in addition, summary 
data for the neutron's history to date are computed, such as 
total distance traveled along the ^-axis during both colli- 
sions, total energy lost during both collisions, and direction 
cosine with the ^r-axis. In this manner the neutrons are car- 
ried from collision to collision, until all of them either are 
absorbed in undergoing a collision, or drop through succes- 
sive collisions, below a stipulated energy level, at which 
point they are no longer useful for the purposes of the 
problem. 

In the sequence of calculations described here, the energy 
loss, u, and the direction cosine, fi (both of which are inde- 
pendent of distance), will be calculated first for each colli- 
sion of all neutrons. The distance, s, which is directly de- 
pendent on [I and indirectly upon u, is then calculated for 
successive collisions of each neutron ; s could be calculated 
at the same time that u and ^i are calculated, if the capacity 
of the machines used so permitted, or u, fi, and s might be 
calculated in separate, successive operations in that order if 
machine capacity is limited. It is understood that in actual 
practice many of the operations which will be described as 
separate steps should be combined; the method of combina- 
tion will depend on the machines available, the training of 
the operator handling the problem and, of course, the par- 
ticular problem involved (magnitude of numbers, etc.). 

For this particular example, one thousand neutrons will 
be traced in a medium of two constituents. An elastic scat- 
tering as the result of a collision with an atom of one kind 
will be designated a type 1 collision, with the other type of 
atom a type 2 collision. Any collision resulting in absorption 
will simply be called absorption, without regard to type of 
atom hit. 

Three decks of cards are assembled for the preparation 
and actual computation of the problem described. They are : 
. 1. Probability cards (card layout 1) 

2. Master cards (card layout 2) 

3. Collision cards (card layout 3) 

*This paper was presented by Edward W. Bailey. 
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Card No. 1 

Source 

Ty Card No. 1 

2. 

3. 

4. 

5. 

6. 

7. 

8. 

9. 
10. 
11. 
12. 
13. 
14. 
15. 
16. 
17. 
18. 
19. 
20. 
21. 
22. 
23. 
24. 
25. 
26. 
27. 
28. 
29. 
30. 
31. 
32. 
33. 
34. 
35. 
36. 
37. 
38. 
39. 
40. 



CARD LAYOUT 1 
Title : Probabiuty Card 

Card Color : Solid Blue 

: Probability Tables Computed for 

the Particular Problem 

41. 
42. 
43. 
44. 
45. 
46. 
47. 
48. 
49. 

m 

51. --\(-f-.xxx) 

52, 

53. 

54. 

55. — m(-(-xx.xxx) 

56. 

sr 

59. 
60. 
61. 
62. 
63. 
64. 
65. 
66. 
6L 



69. 

70. 

727 

73 

74, 

2U 

76. 
77. 
78. 
79. 
80. 



■Ai(u) (.xxxx) 
'Ao(u) (.xxxx) 



A^{u) is the probability of a collision of type 1 or of ab- 
sorption. Aq{u) is the probability of absorption. The proba- 
bility of a collision of type 2 is 1.0000— .-^^(m). All values 
on the card are key punched from tables. 

Probability Cards 

The probability cards (card layout 1), of which there are 
some twenty or thirty, are key punched from probability 
tables calculated for the particular problem at hand. Each 
card contains, in columns 53-57, a value u which is the 
lower value of the energy range which the card represents, 
and in columns 68-71 and 72-75 the probabilities associated 
with the particular energy range, of absorption or scatter- 
ing with the different types of particle involved. Thus, Ai{u) 
is the probability of a type 1 scattering or of absorption, 
and Ao{u) is the probability of absorption. The probability 
of a type 2 scattering is then 1.000 — .^^(m). Probability 
card 1 also contains the value A which is the mean free path, 
or probability of scattering within a given distance, asso- 
ciated with the energy range. 



CARD LAYOUT 2 

Title : Master Card 

Card No. 2 Card Color : Solid Red 

Source : Various Tables and Computations 



(1) Xh Card No. 2 
2. 
3. 
4. 
5. 
6. 
71 



(2) 8. 
9. 

HL 
11. 

(3) 12. 

w. 

(4) 15.- w( 
16^ 



— K (-(-.xxx) 



:.xxx) 



30, 
(9) 31, 
32J 
33. 
34. 
35. 
36. 
37. 
38. 
39. 
40. 



(5) 18. — Vl-«M+.xxx) 

12.. 
20. 

23, 
24, 
(1) 25.1-/. ( 
26, 
27. 



cos TTfa (itx.xxx) 



.xxx) 



(6) 28. — VI— M^* (+.XXX) 
29. 



• Inn ( — x.xx) 



41. 
42. 
43. 

4!!l 

46. 
(7) 47. 
48. 
42J 
50. 
51. 
52. 
53. 
54. 
55. 
56. 
57. 
58. 
59. 
60. 
61. 
62. 

(1)P 

69. 
70. 

(1) 73. 

(1) 76, 

(1) 79. 
80. 
r, = r, = r 



■irr» (xx.xxx) (re- 
duced for our 90° 
deck) 



- Am (-|-x.xxx) 

- Collision Code 

• Ti (.xxx) 

rs (.xxx) 

— Vi (.xxx) 



The number of the operation in which a field is derived 
is given in parentheses to the left of the card columns. See 
"Master Cards" Operations for step involved. 

Master Cards 

The deck of master cards, of which there are 2,000 in 
this example (card layout 2), contains all the tabled val- 
ues, such as functions of angles and natural logarithms, 
which must be searched from time to time during the prob- 
lem, as well as some calculated values which appear over 
and over again, and hence can be intersperse gang punched 
more readily than calculated each time. The arguments on 
the master cards are {i, r^, r^, and r^, and on any one card 
the four arguments are the same: they are put in the four 
different fields as a convenience to the operator, the fields 
on the master card being then lined up with those on the 
collision card. Since all of the arguments are three-digit 
numbers, our master deck will consist of 1,000 cards repre- 
senting the 1,000 different choices of a three-digit number 
(000-999, inclusive) and having a collision type code of 1, 
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and another 1,000 cards representing all possible random 
digits, having a collision code of 2 (refer to master card 
table, step 1). In addition to the 4 three-digit arguments, 
the master card will also contain the following which are 
numbered in accordance with the step number on the table 
of operations referring to the master card. 



CARD LAYOUT 3 (Continued) 



Step 
No. 



Function 



2 K"" = 1 - ^^M{M + l)-^ where the value of M is 
a constant depending on the collision type, is calculated 
and punched in the master cards ; 

Z K h then intersperse gang punched into the master 
cards from a square root deck ; 

4 c = [(M + l)i^ - (M - l)/K]/2 is calculated on 
the 604 and punched; 

5 VI— w^ is intersperse gang punched from a sin cos 



deck; 



6 yi—fj? is intersperse gang punched from a sin cos 
deck; 

7 ^3 is multiplied by tt, reduced to an angle less than 90°, 
the absolute value of whose cosine equals that of itr^, 
and the angle is punched in the card while the sign is 
punched in column 23 ; 

8 cos Trrg is intersperse gang punched into columns 20-23 
from the sin cos deck ; 

9 In r^ is intersperse gang punched from an In deck ; 
10 Am = — In ^^ is intersperse gang punched from an In 

deck. 
This completes the data on the master cards. It should be 
noted that a number of the steps could be combined in ac- 
tual computation, such as steps 1, 2, 4, and 7. Also, in the 
absence of suitable function decks, some of the functions 
could be calculated instead of intersperse gang punched. 

CARD LAYOUT 3 

Title : Coi.i,ision Card — 

Basic "Random Number" Card 

Card No. 3 Card Color : Plain Manila 

Source : Computations and Random Number Tables 



;.xxx) 



(1) 2- 


Card No. 3 


207 




2. 
3. 




(2)|; 


— cos Trrs (±x. 


4. 




23. 




5. 




24 


6. 




25. 


7. 




26. 


8. 




27. 


9.- 




28. 


10. 




29. 


11. 




W\ 




12. 




(3) 31. 


— Inn ( — x.xx 


13. 




32. 




14. 




33. 


15. 




34. 


16. 




35. 


17. 




36. 


18. 




37. 


19. 




38. 





40. 


61. 


41. 


62. 


42. 


63. 


43. 


64. 


44. 


65. 


4t:i 


66. 


46. 


,(?7, 


(1)47. 


— Serial Number 68. 


48. 

49, 


^^^ 70. 


50. 


Zk 


51. 


72. 


52. 


(1)73. 


53. 


74. 


54 


i75. 


55. 


(1) 76. 


56. 


77. 


57. 


W 


58. 


(1)79. 


59. 


80. 



Tx (.xxxx) (to deter- 
mine collision type) 



( I ) 73. 1 rz (.xxx) (to determine 

energy loss) 

(1) 1§-\—^3 (.xxx) (to determine 
azimuth) 

Ti (.xxx) (tO' determine 
distance traveled be- 
tween collisions) 
The random numbers r-y, r^, r^, and r^ are independent of 
each other. The number of the operation in which a field is 
derived is given in parentheses to the left of the card col- 
umns. See "Collision Cards" Operations for step involved. 

Collision Cards 

The collision cards (card layout 3) are begun as "ran- 
dom number" cards. An estimate is made, on the basis of 
how many neutrons are involved and how many collisions 
are expected, of the number of collision cards which will be 
needed. 

The following steps are then taken to obtain the infor- 
mation required concerning the case history of each 
neutron. These steps are numbered in accordance with 
the steps listed on the table of operations for the collision 
cards. 

Step 

No. function 

1 The estimated number of collision cards required is 
made up with calrd number (3) in column 1 and ran- 
dom digits, which will be the basis for the random 
choices to be made for each neutron, in columns 68- 
80. The first four of the random digits are called r^, 
the next three rg, the next three r^, and the last three 
r^. The source of the random numbers, which may be 
either taken from tables or calculated, should be 
recorded to avoid choosing them in the same way in 
a future problem of the same nature. The cards 
should be numbered serially to preserve the order. 

2 Since cos irr^ and In ^4 are dependent only on random 
numbers, they may be put in all of the basic collision, 
or "random number," cards before calculation is 
started. The cards are sorted to order of r^, match- 
merged with the master deck, and cos Trrg intersperse 
gang punched into the random number cards. 

3 Similarly, In r^ is intersperse gang punched from the 
master deck into the random number cards. The ran- 
dom number, or basic collision, cards then look like 
card layout 3 and are ready to be developed into com- 
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plete collision cards. They are next sorted back to 
serial number order to restore the randomness of the 
random numbers. 

4 The first 1,000 of the random number cards are num- 
bered consecutively from 000 to 999 in columns 2-4 ; 
01 emitted into columns 5-6; O's emitted into field 
53-57; and fx = I - 2ro, where r^ is chosen from the 
other random numbers on the card, is calculated and 
punched in columns 24-26. Each of these cards will 
then represent one of the thousand neutrons as it 
undergoes its first collision, having an initial energy 
loss u of zero and having a random direction cosine /x. 

5 The thousand cards are sorted to order of ^, match- 
merged with the master deck, and Vl-/^^ intersperse 
gang punched into the collision cards. 

6 The cards are then sorted into ascending order of the 
value of energy loss, u, in columns 53-57 (which will, 
of course, be zero for all cards for the first collision, 
but not thereafter) and match-merged with the proba- 
bility cards on columns 53-57, so that for each energy 
range there is a probability card followed by all the 
collision cards in the particular range. Collision cards 
having an energy level less than the specified limit 
are given a collision code of 3 and set aside, since they 
will undergo no more collisions. X. is intersperse gang 
punched from the probability cards into the remaining 
collision cards. 

7 The merged deck of probability cards and collision 
cards is run through the collator, reading from each 
probability card the probability Aq{u) of absorption 
and selecting from the following collision cards all 
cards whose r^ < Aq{u). These cards are coded in 
column 67 and set aside, since they will undergo no 
more collisions. The remainder of the collision cards, 
which are still merged with the probability cards, are 
recollated, this time comparing r^ with ^^^(m). Colli- 
sion cards whose r^ ^ A^(u) are coded 1 in column 
67 to indicate a type 1 collision, while cards whose 
r^ > A^{u) are coded 2 to indicate a type 2 collision. 

8 Collision cards having a code of 1 or 2 are sorted to 
the order of ^2 by collision code (67, 72-74) and 
match-merged on those columns with the master deck. 
K^, K, w, Vl-<^^ and Aw (all of which depend directly 
on the collision type and rg) are intersperse gang 
punched from the master cards into the collision 
cards. The master deck is then put together again 
while the collision cards are being sorted to the order 
of neutron number (for convenience in case a particu- 
lar neutron has to be checked in the future). 

9 A random number card (card layout 3) is merged 
behind each collision card to become the second colli- 
sion card for that particular neutron. 

10 The merged deck is put in the 604 and fxi, V1-/4, <^u 
Vl-(4, cos Trrg. are read, where i is the collision num- 



ber; /xj + i = ju,i <«){— \/l-/4 Vl-<"i cos ir r^^ is calculated 
and punched in the random number card following 
the collision card. The collision number is also read 
from the collision card, increased by one, and punched 
in the following card. The neutron number is carried 
forward directly from the collision card to the follow- 
ing random number card, whereupon the random 
number card becomes the new collision card for the 
neutron. Mj and Am are read from the ith. collision 
card, added together and punched as W{ + 1 in the ith. 
card and as Ui in the i -\- 1th card; that is, the total 
energy loss after the ith collision is the same as the 
total energy loss before the i -\- 1th collision. 

1 1 The cards are then sorted on the collision number, the 
first collision deck set aside temporarily, and the deck 
of second collision cards taken through steps 5 to 11, 
thus generating a deck of third' collision cards, and so 
on until all the neutrons have disappeared. 

There is, then, a collision card for each collision each 
neutron has undergone. This card contains the total 
energy loss of the neutron after the collision and its 
direction cosine. To complete the data only the total 
distance traveled after each collision is needed. 

12 To get this value all the collision cards are merged or 
sorted into ordeT of collision number within neutron 
number (columns 2-4, 5-6). The first collision card for 
neutron 000 is read, and p = Aln r^ and /\s — pfx are 
calculated, p and ISz are punched in the first collision 
card ; A^' is punched in the ^^ field in the first collision 
card and stored as well. 

13 The second collision card for the same neutron is 
read, p and A^ calculated and punched, A^ added to 
the z stored from the previous collision and the new 
2 value punched in the second collision card ; that is, 
for any given neutron s'i+i = A^i+i -f- ^i, where i is 
the collision number. When the z values are com- 
puted, calculation of the data pertaining to individual 
collisions of each neutron is completed (card layout 
4). These data can then be grouped and combined 
for whatever statistical studies are desired. 

Complete case histories for each neutron are readily 
available and may be used in the solution of problems of 
various boundary conditions involving the constituents of 
the specific problem. Final compilation of the data takes 
the form of frequency distributions and factorial moments 
of these distributions. Two examples of such distributions 
are: 

1. Distribution of the total distance traveled from the 
source before absorption or reaching a given energy 
level. 

2. Distribution of the energy loss at each collision. 
The factorial moments for these distributions are com- 
puted for curve fitting and calculation of various param- 
eters of these distributions. Calculation of these moments 
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involves the use of standard accounting machines and 
presents no problems. 

CARD LAYOUT 4 

Title : Coi.i,ision Card 

Card No. 3 Card Color : Plain Manila 

Source : Card 3 

(Random Number Card, see Card Layout 3) 



(DJ. 

2. 

(4) (10) 3. 

M 

(4) (10) ^ 

71 

(8) I 

m 

(8)12, 

IT 

(8)15 
16,. 

m 

(8)18. 
12J 
20. 

(2)|- 

23. 
24. 
(4) (10)25. 
26.. 

m 

(5)28. 
29^ 
30. 

(3)31. 
32.. 

34. 

(12)36. 

3a 

(12)39. 

40. 



h Card No. 3 
. Particle No. 

• Collision No. 
'K2 (+.XXXX) 

■K (+.XXX) 

• w (dz.xxx) 



(12) 



— Vl-«2 (+.XXX) 

cos Trrs (±x.xxx) 
fi (±.xxx) 



-VI— /^2 (±.xxx) 
•In n (— x.xx) 

■ p (x.xx) 
•A^ (±x.xx) 



.1^69. 

^^^70, 



. s (±xx.xx) 



45: 

46. 

(1)47. 

48. 

w 

(6)51. 
52, 
53. 
54. 
(4) (10)55. 
56. 
57^ 

59. 

(10)60. 

61,. 

62. 

66. 
(7)6p— Collision Code 



— Serial No. 
•X (-|— xxx) 
Ui (-)-xx.xxx) 

•Ui+r (-|-XX.XXx) 

• Am (+x.xxx) 



Type of Cards : Master Cards 
Operation 



n (.xxxx) 
71.. 

m 

(1)73. —r2 (.xxx) 

21 

75. 
(1)76.1— rs (.xxx) 

78. 
(l)79.h-n(.xxx) 
80, 

Formulation 



The number of the operation in which a field is derived 
is given in parentheses to the left of the card columns. See 
"Collision Cards" Operations for steps involved. 

DISCUSSION 

Mr. Turner: I am not familiar enough with the details 
of this calculation to know whether it is possible to go back 
and reconstruct, shall we say, the x coordinates. Can that 
be done? 

Mr. Bailey: Not in this particular problem, I am not 
very familiar with the work which is being done now. It 
may be that one of those actually doing the work could 
answer that question. 

Miss Johnson: We can go back on this problem and re- 
construct the X coordinates but it wouldn't be practical. We 
are working with the problem now where all the diflferent 
coordinates, and time, are being calculated as we go, and 
we are using six constituents or six different types of atoms 
instead of only two. 

Chairman Hurd: In this problem there was no interest 
in the x coordinates, Mr. Turner. 

Mr. Turner:! was thinking of its use to get the spectrum, 
that is, the scattering, for the lower energy particles. If you 
could get the x coordinates, you could rotate your space 
and from the same data get the distribution for lower 
energy particles. 

Mrs. Dismuke: That is the idea, of course, in the prob- 
lem we are doing now. We started at higher energies. In 
the particular problem, which Mr. Bailey described, we 
started at such a low energy that our population probably 
wouldn't be big enough. 



Machine Operations Involved 



1. Make Basic Master 
Cards 



2. Calculate and Punch i^^ k^ =, i - Ar^M{M + 1)- 



3. Punch K in Cards 



4. Calculate and Punch (0 oi={{M-\-l)K- {M-\)/K]/2 



5. Punch VI ~ w^ ill 
Cards 



On 604 generate the consecutive numbers 000-999, 
punching the number for each card in columns 24-26, 
72-74, 75-77 and 78-80. Emit a "2" into column 1 and 
a collision code of "1" into column 67. Make a similar 
deck with a "2" in column 67. 

Calculate on 604, controlling on collision code to emit 
proper value of M. 

Sort the cards to the order of K^, match-merge with a 
square root deck, and intersperse gang punch K into the 
master cards. 

Calculate on 604, controlling on collision code to emit 
proper value of M. 

Sort the cards to the order of <a, match-merge with a 
sin-cos deck (matching w with the sine) and intersperse 
gang punch ^/l — <a'^ (cos) into the master cards. 
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Machine Operations Involved 



6. Punch yi— ju,^ in 
Cards 

7. Calculate and punch 
TT ^3 so that we can pull 
cos w ^3 from our 90° 
deck. 

8. Punch cos 7rn in Cards. 



9. Punch In r^ in Cards. 
10. Punch Aw in Cards. Aw 



= -InK' 



Same operation as above, except match fx instead of <o 
with sine. 

Calculate -n-rg directly if r^ ^.500; calculate 180° —Trr^ 
if ^3 >.500 and punch X in column 23 to indicate a 
negative value. 



Sort to the order of Trr^, match-merge with a 90° sin-cos 
deck and intersperse gang punch cos vr^ into the master 
cards. 

Sort to the order of r^, match-merge with In deck and 
intersperse gang punch In r^ into master cards. 

Sort to the order of K'^, match-merge with In deck and 
intersperse gang punch In K- into master cards, omitting 
sign since In is negative and we want —In K'^. 



Type of Cards : C01.1.1S10N Cards 



1. 



2. 



3. 



4. 



Make Basic Collision, 
or "Random Number," 
Cards. 



Punch in each "Ran- 
dom Number" Card the 
Cosine of a Random 
Angle of Deflection. 



Punch on each "Ran- 
dom Number" Card a 
In picked at random. 



Pick 1000 neutrons to 
undergo their first colli- 
sion with no previous 
energy loss, and start 
them in a random di- 
rection. 



Punch ^/l—|x^ in the 
Collision Cards. 



6. Punch A, or the mean 
free path, in the cards. 



cos irr.^ 



In r. 



Collision no. = 01 

u = 00.000 

/x = 1 — 2^0 for first collision only 



A is the probability of scattering 
in a given range and is dependent 
on the energy range into which 
the neutron falls. 



Determine approximately how many collision cards will 
be needed and reproduce, or calculate, random numbers 
into columns 68-80 of that many blank cards. At same 
time, emit a "3" into column 1 of these cards and num- 
ber the cards serially in cols. 45-49. Call cols. 68-71 r^, 
72-74 r„ 75-77 r,, and 78-80 r,. 

Sort "random number" cards to the order of r^ in cols. 
75-77, match-merge with our master deck on cols. 75-77, 
and intersperse gang punch cos -n-r^ into the random 
number cards. (Use only half of the master deck: the 
collision code 1 or the collision code 2 cards, since cos 
TT^s is independent of collision type.) 

Sort "random number" cards to the order of r^ in cols. 
78-80, match-merge with half of the master deck (say, 
the collision code 1 half) on cols. 78-80 and intersperse 
gang punch In r^ into the random number cards. Sort 
the random number cards to the order of cols. 45-49 to 
restore randomness. 

On the 604 generate the consecutive numbers from 000- 
999, punching the number for each card in columns 2-4. 

Emit "01" into cols. 5-6 

Emit "00.000" into cols. 53-57 
Choose ^0 from r^, r2, r^, and r^ : say the third digit of 
each number, and calculate fi = 1 — 2rQ. {/x will be chosen 
in this fashion only on the first collision cards.) 

Sort the collision cards to the order fx, match-merge 
them with the master deck, and intersperse gang punch 
VI— i^^ into the collision cards. 

Sort the collision cards to the order of u (which will be 
00.000 on the collision cards for the first collision but not 
thereafter), merge them with the probability cards on 
columns 53-57 so that, in the merged deck, there will be 
a probability card for a certain energy range, and behind 
it will be all the collision cards with energy Ui within 
that range, then the next probability card and the colli- 
sion cards within its range, etc. Check the sequence of 
the merged deck and then intersperse gang punch A 
from the probability cards to the collision cards. Do not 
sort the probability cards and collision cards apart. 
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Machine Operations Involved 



9. 



10. 



Determine whether each 
neutron (a) fails to un- 
dergo another collision 
because of low energy ; 
(b) is absorbed; (c) 
undergoes a collision of 
type 1 or (d) under- 
goes a collision of type 
2. If not (a) then (b), 
(c),or (d) is a random 
choice. 



(a) If energy loss, u, of the neu- 
tron exceeds a certain value the 
neutron undergoes no more colli- 
sions ; (b) if r^ < A^ the neutron 
is absorbed; (c) if A^ '^ r^ '^ 
/^o the neutron undergoes a type 
1 collision; (d) if r, > A^ the 
neutron undergoes a type 2 colli- 
sion. 



Cards of group (a) are removed from the deck by hand 
and "3" punched in them as the collision code. Opera- 
tions (b), (c), and (d) are all done on the collator. 
Cards falling in group (b) are given a collision type 
code of "0"; cards falling in group (c) are coded "1"; 
and cards falling in group (d) are coded "2." Collision 
cards having a code of "3" or "0" are removed from the 
deck since the neutrons these cards represent will un- 
dergo no more collisions, while cards having a code of 
"1" or "2" are carried on through the collision. 



Pick a random energy Use fg and collision code for ran 

loss for each neutron dom choice of 

and associated with this, K^ =^l—4r^M{M-\-l)-^ 

0>=:[(M-^1)K-{M 

Am = — In K2 



a value of w and V 1 



Pick a set of random 
numbers to be used in 
calculating the next col- 
lision cards for each 
neutron. 



Sort collision cards to order of ^2 by collision code (cols. 
67, 72-74), match-merge with the master deck on 
columns 67, 72-74, and intersperse gang punch K^, K, 
1 )/^]/2, to), V 1 ~ «^ and Aw from the master cards to the collision 
cards. Sort the collision cards to order of neutron num- 
ber (for future convenience) while putting the master 
deck back together. 

Merge a "random number" card (see card layout 3) 
behind each collision card, the random numbers on the 
card to be used in calculating data for the next collision. 



a. Calculate in + ^ 
the next collision. 



for 



/*i+ 



b. Punch the neutron 
number and new colli- 
sion number in the new 
card. 

c. Calculate w^ + ^ and 
punch it in both the old 
and the new collision 
cards for each neutron. 

11. Sort collision cards on 
the collision number 
and repeat steps 5-11 
until all neutrons have 
disappeared. 

12. Pick a random distance 
p for each neutron to 
travel and calculate the 
distance A^ traveled 
along the 5^-axis. 

13. Repeat operation 12 for 
all collisions of all neu- 
trons starting each neu- 
tron at 5f = 0. 



V 1 — wf COS TT ^3 



Coll. No. i+\ = Coll. No. i -f 1 



— Ui 



+ Am 



Read /*, w, y\ — o)^, yi—fj?, cos -nr^ from the old colli- 
sion cards, calculate /xj+i and punch it in the new colli- 
sion cards. 

Intersperse gang punch the neutron number from the 
old collision card to the new collision card. Read the 
collision number from the old card, increase it by one, 
and punch it in the new collision card. 

Read Ui and Am from the old collision card, calculate 
Wi+i and punch it in columns 58-62 of the old collision 
card and in columns 53-57 of the new collision card. 



p = \lnr^ 
Az = p/x 

Zui = As-\- Si 



Sort or merge cards to order of collision number by 
neutron number (cols. 2-4, 5-6). Read first collision 
card for first neutron; calculate and punch p, /\s, and s, 
storing z. Read second collision card for first neutron ; 
calculate and punch p, As, and s, etc. 



A Nlonte Carlo Nlethod of Solving Laplace's Equation 



EVERETT C. YOWELL 

'National Bureau of Standards 



DURING the first meeting of this seminar we discussed 
the solution of Laplace's equation in two dimensions by 
smoothing techniques. As was pointed out at that time, the 
smoothing process is only one approach to the problem. It 
was an approach that was being tested at the Institute for 
Numerical Analysis because it used a simple, iterative rou- 
tine that a calculating machine could easily be instructed to 
follow. 

A second experimental calculation that we performed, in 
seeking a simple method of solving Laplace's equation, was 
a test of a method suggested by Dr. Feller. The basis for 
this method is completely different from the other methods 
we mentioned. Dr. Feller was seeking a probability ap- 
proach to the problem. That is, some sort of a random 
process is set up such that the probability distribution of 
the answer given by the process obeys the same differential 
equation as that of the physical problem. It is to be hoped 
that the random process offers a simpler computing scheme 
than any of the direct approaches to the solution of the 
differential equation, thus making the computational task 
simpler and less time-consuming. 

In the case of Laplace's equation in two dimensions, this 
random process is merely a two-dimensional random walk. 
In such a walk, uniform steps are made along one or the 
other of the coordinate directions, the choice being purely 
random, and in either a positive or a negative direction, the 
choice once again being random. If such a walk is started at 
a point inside the boundary of a region, it will eventually 
end up at the boundary. The number of steps will vary for 
two different walks starting at the same point, but the aver- 
age number of steps can be computed. Suppose, now, a 
large number of walks from a single interior point is made. 
Each time a boundary is reached, the process is stopped, 
the value of the function on the boundary is recorded, and 
the process is repeated from the same starting point. If, 
after a large number of walks, an average of all the bound- 
ary values is noted, that average will approximate the value 
of the function at the starting point and will converge to 
that value as the number of walks increases to infinity. 

This process was tested on the IBM Type 604 Electronic 
Calculating Punch. A region bounded by the four lines 
X — y = Q and ;ir = y = 10 was selected and a unit step 
was made in the random walk. The boundary values were 
/(10,y) = /(.r,10) = 0, /(0,y) = 100 - lOy, /(;r,0) = 
{\0 — xy. 



The random variables were introduced into the problem 
according to the evenness or oddness of the digits in a table 
of random digits prepared and furnished us by RAND 
Corporation. 

The wiring of the 521 control panel was very simple. The 
machine was to make two choices, depending on the even- 
ness or oddness of two random digits. Hence, two columns 
of the random digit cards were wired through two digit 
selectors, and the outputs of all the even digits of each 
selector were wired together. These two even outputs were 
then used to transfer two calculate selectors. 

The initial value of the coordinates of the starting point 
was introduced by a special leader card. This carried a con- 
trol punch in column 79, and the original coordinates in 
columns 1-4. These values were read directly to factor stor- 
age units 1 and 3 and also to general storage units 1 and 3, 
and these units were controlled to read only on an x punch 
in column 79. 

The final value of the answer was punched on a trailer 
card, which carried a control punch in column 80. The sum 
of the boundary values, and a tally of the number of times 
the boundary was reached, were read out of general storage 
2 and 4 and punched in the trailer under control of the y 
in column 80. 

The wiring of the 604 control panel was more compli- 
cated. The analysis chart is given below. 

Factor Storage MQ Counter General Storage 

12 3 4 12 3 4 

X y 

Read 

Step Suppress Operation 

1. Counter RI -|-, GS 1 read out if calc selec- 
tor 1 energized, GS 3 read out if not. 

2. Emit "1," RI 2nd, counter add if calc selec- 
tor 2 energized, subtract if not. 

3. Counter RO. GS 1 RI if calc selector 1 
energized, GS 3 RI if not. 

4. Emit "1," RI 3rd, counter subtract. 

5. Balance Test. 

6. N. Counter read out and reset. 

7. N. GS 2 read out. 

8. N. Emit "1," counter add. 

9. N. Counter read out and reset, GS 2 read in. 

10. N. FS 1 read out, GS 1 read in. 

11. N. FS 3 read out, GS 3 read in. 

12. P. Emit "1," RI 3rd, counter add. 

13. P. Emit "1," counter subtract, balance test. 
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Read 

Step Suppress Operation 

14. P. Counter read out and reset. 

15. P. Emit "1," counter add, RI 3rd. 

16. P. Counter subtract, RO GS 3 if calc selector 

1 energized, RO GS 1 if not. 

17. P. Read out and reset counter, MQ RI. 

18. P. Emit "1," RI 3rd, counter add only if calc 

selector 1 energized. 

19. P. GS 3 RI, counter RO and RS if calc selec- 

tor 1 energized, GS 3 RI, MQ RO if not. 

20. P. Multiply, GS 3 RO. 

21. P. ^ adjust, RI 2nd. 

22. P. RO GS 4, RI 3rd, counter add. 

23. P. RO and RS counter, GS 4 RI, RO 3rd. 

24. P. GS 2 RO, counter add. 

25. P. Emit "1," counter add. 

26. P. Counter read out and reset, GS 2 RI. 

27. P. FS 1 read out, GS 1 read in. 

28. P. FS 3 read out, GS 3 read in. 

29. Counter read out and reset. 

The first three steps compute the coordinate of the next 
point in the random walk. Either the x or the y coordinate 
is adjusted, according to one of the two random digits. And 
the adjustment is either positive or negative, according to 
the other random digit. Steps four and five test to see if the 
upper bound in ;r or 3; has been reached. If a negative bal- 
ance test occurs, then the point is still under the upper 
bound, and steps 6 through 11, which correct the tally and 
reset the coordinate units to their original values, must be 
suppressed. Steps 12 and 13, which test for the lower 
boundary are permitted to take place, and the new balance 
test supersedes the old one. Now a negative test is a sign 
that the boundary has been reached ; so steps 14 through 28 
— which compute the value of the function at the lower 
bound, correct the tally, and reset the x and y storage units 
to their original values — are suppressed on a plus balance 
test. 

Three possible conditions can arise from the balance 
tests. If the first test is positive, the upper boundary has 
been reached. Then steps 6 through 11 occur, and the re- 
maining steps are suppressed. If the first test is negative, 
the upper boundary has not been reached, and a second test 
must be performed to see if the lower boundary has been 
reached. So steps 6 through 11 are suppressed, and steps 12 
and 13 occur. If the balance test on step 13 is positive, the 
lower boundary has not been reached, and steps 14 through 
28 are suppressed. If it tests negatively, the lower boundary 
has been reached, and steps 14 through 28 occur. In all 
cases, step 29 is taken to reset the counter at the end pf 
each computation. 

The operating procedure was as follows : From the com- 
putation of the mean length of the random walk, an estimate 
was made of the number of steps needed to give a specified 
number of walks. This many random digit cards were se- 
lected, a leader card was put in with the coordinates of the 
starting point, and a trailer card in which the answer was 



to be punched. This deck was then run through the 604, 
and the sum of the boundary values and the tally of the 
number of times the boundary was reached was punched in 
the final card. The quotient of these two numbers gave the 
value of the function at the starting point. 

The tests indicate that the method will give the correct 
answers, but the speed of convergence is very slow. The 
smoothing method converges as (1/w), where n is the 
number of times the field is smoothed. In this- case, we 
mean that the diflference between the approximate solution 
and the true solution is proportional to (l/w). But in the 
Monte Carlo Method, the convergence is proportional to 
(1/Vw). And the convergence here is a statistical con- 
vergence ; that is, the probable error is proportional to 
(1/Vw), where n is the number of random walks. With 
statistical convergence, one can never guarantee that the 
solution is correct, but one can state that the probability is 
high that the solution is correct. 

It is obvious that the control panels described will not be 
applicable if the boundary values of the function are at all 
complicated functions of the coordinates. The method can 
still be used if the following changes are made: A trailer 
card is inserted behind each random digit card, and the co- 
ordinates of the point are punched in the trailer card. Or, 
similarly, a few columns of the random digit deck are re- 
produced into work cards, and these are run through the 
604, punching the coordinates of the point in each card. 
The boundary values can be internally tested as long as the 
boundary is a rectangle, for two sides can always be made 
zero by a suitable translation of axes, and the other two 
constants can be stored in the 604. Hence, the x and y 
storage units can be reset to their initial value whenever 
the boundary is reached. If the boundary is not rectangular, 
then the mean length of the random walk and the dispersion 
of the mean length can be computed and enough random 
digit cards assigned to each random walk so that any pre- 
assigned percentage of fixed length walks will cross the 
boundary, A sufficient number of walks are made so that a 
reasonable number of boundary crossings are available. 

The cards from the 604 in either of these methods will 
contain the coordinates of the points of the walk. In the case 
of rectangular boundaries, the boundary point can be found 
by a sorting operation. In the case of the non-rectangular 
boundary, the sorting operation must be followed by a 
selection of the first point of each walk that crosses the 
boundary. In any case, the functional value at each bound- 
ary point can then be gang punched in the proper boundary 
cards and an average made. 

The great drawback of this statistical method is its slow 
speed of convergence. This should not <;ause the method to 
be discarded, for the ideas of Monte Carlo procedures are 
still new and ways may still be found to speed up the con- 
vergence of the solution. It is also true that the speed of 
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convergence of the Monte Carlo Method is not affected by 
the dimensionahty of the problem, and this may prove to be 
a very great advantage in problems involving three or more 
dimensions. Finally, Monte Carlo procedures may be very 
important in problems where a single number is required 
as an answer (such as the solution at a single point, or a 
definite integral involving the solution of the differential 
equation). In these cases the entire solution would have to 
be found by smoothing techniques. With these limitations 
and advantages recognized, the simplicity of the procedure 
certainly makes the Monte Carlo Method worth considering 
for the solution of a partial differential equation. 

DISCUSSION 

Mr. Turner: When punching the position of each succes- 
sive point in a random walk from a given starting point (in 
order to find the value of the function at the given point) 
can one consider the particular boundary value as one term 
of the partial sums for each of the points that was crossed 
in the random walk? That is, would it be a safe process 
with a large number of walks, to consider each one of the 
points, which was crossed as a starting point, for one of the 
other points in the problem ? 

Dr. Yowell: This question is now being investigated by 
Dr. Wasow and Dr. Acton at the Institute for Numerical 
Analysis. Their preliminary findings, about which I hesi- 
tate to say very much, indicate that this can be done, but it 
raises the standard deviation by a great deal, although it 
gives the same mean. 

Mr. Turner: Another suggestion is that instead of start- 
ing from the interior, start from the boundary; then the 
problem of when the boundary is reached does not arise. 
In other words, use the symmetry of the functions. Start 
from the boundary and walk into the interior. 

Professor Kac: That defeats the purpose. The result is 
the harmonic function at the starting point. So if one starts 
at the boimdary, then one gets what he already knows, be- 
cause the boundary value is given. 

Mr. Turner: What I meant was: start at the boundary 
and then generate a sequence of points that are associated 
randomly with that particular boundary point (a sequence 
of interior points). In the final operation sort the cards on 
the point coordinates, run them through the accounting 
machine, and sum the values that are on the boundary 
cards; in other words, gang punch each boundary value 
into each one of these cards that was generated by the 
random walk. 

Dr. Yowell: The question that occurs to me is, if one 
starts at any two boundary points, is the relative frequency 
of reaching a particular interior point the same as the rela- 
tive probability of reaching the two boundary points by 
random walks originating at the particular interior point? 

Professor Tukey: It seems to me that Mr. Turner has a 
very interesting suggestion, and that is to start at the 



boundary and walk n steps inside, until the boundary is 
reached. One then has two n pairs of numbers, each pair 
consisting of a boundary point and an interior point, and 
I understand the suggestion is to take a lot of walks of this 
type and cross-tabulate the number of times a pair (consist- 
ing of a certain boundary point and a certain interior point) 
occurs. It is an interesting speculation. 

Professor Kuns: I would like to remark that we have 
taken a lot of liberties here. These probabilities around the 
edges (the Green's functions, which we are calculating 
here) must be evaluated for every interior point, n^ in num- 
ber ; they must be multiplied in each case by every boundary 
value. An, and that is too much work already. 

Mr. Bell: Consider doing the problem on a type 604. The 
standard 604, of course, won't program repeat, but it has 
been pointed out that internal wiring can be altered to 
cause the machine to repeat the program cycles indefinitely. 

The 604 has a pulse rate of 50,000 pulses a second. It 
takes about 25 pulses to do a particular operation, except 
multiplication and division, and they take a few more. Sup- 
pose we assume 100 cycles ; which means 2,500 pulses, so 
that 20 of these testing operations a second can be per- 
formed. We can thus make about 1,200 such tests in a 
minute. 

Thus, my question is : Would it be reasonable to take a 
604, and, with quite minor modifications, load it up, let it 
run these random walks where all the computing is being 
done down to reaching a boundary condition, at which point 
a new card feeds and the computing continues ? 

Here is an opportunity to get the speed that is really 
available with electronic computing. I am certain that the 
circuits that would have to be changed are minor, a matter 
of a few hours' work, and a standard 604 could do this. 

Mr. Sheldon: If this is to be done, it will be necessary to 
make the 604 compute its own random numbers since the 
random numbers must get into the machine in some way. 

Mr. Bell: I wonder if that is a limitation. It is being done 
in the 602-A. It would depend, of course, on the functions ; 
but a great deal can be done with the 604, and people who 
have had little or no experience with it are often amazed at 
how much can be done in spite of the limited storage 
available. 

Dr. Yowell: We have actually tried generating random 
digits on the 604, We have 100,000 of them generated on 
a 5 by 8 representative multiplication, much of it the type 
that has been used on other machines, of squaring a ten- 
digit number and extracting the central ten digits. These 
have been, subjected to the usual tests, and all we can say 
so far is that the digits are extremely lumpy and the ques- 
tion of how homogenized the digits have to be, in order to 
give good results, is something that should certainly be con- 
sidered before generating random digits with a small multi- 
plier or multiplicand. 

Chairman Hurd: I think the idea sounds promising. 
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I HAVE REMARKED that people working with 
computing machines frequently rediscover old mathematics. 
My philosophy is that with a computing machine one can 
forget classical mathematics. It is true that if one knows a 
little analysis, it can be worked in, but it is not necessary. 
However, Professor Kac^ indicates that it looks as though 
I am going to have to learn some new mathematics after 
all, about Wiener functionals, to really understand what I 
am trying to do with the Monte-Carlo Method. 

Mr. Bell has introduced the idea of a "duz" board, which 
is a control panel by which all sorts of calculations can be 
done without having to worry about wiring for each prob- 
lem. The Monte-Carlo Method is really a "duz" method ; 
and can do many kinds of problems without one having to 
think up new schemes of attacking each one. 

In particular, I like to feel that one contribution the 
Monte-Carlo Method has made (and the Monte-Carlo 
Method is probably only the first new method we are going 
to have for computing machines), is the by-passing of 
specialized mathematical formulation, such as differential 
equations. From the viewpoint of a physicist or a chemist 
there really is no differential equation connected with the 
problem. That is just an abstraction that allows the prob- 
lem to be solved by means of the last few hundred years 
of mathematics. With a computing machine one cannot use 
analysis directly. Here is an opportunity (for the physicist 
and the chemist) of putting the real problem right on the 
machine without having to go through differential equa- 
tions. 

Although Professor Kac^ and Dr. Yowell^ have illus- 
trated this much more thoroughly, I would like to discuss 
again the very simplest kind of problem, namely, diffusion. 
We can discuss diffusion in one dimension, although I will 
indicate there is no fundamental difference in approach 
for many dimensions (in contrast with some analytical 
methods). 

Consider a source of material such as a dye in a capillary 
in which the dye can diffuse either way. We wish to find 
out the distribution of these molecules after time,f. We are 
accustomed to say that the concentration varies with time 



according to its gradient in concentration along the x axis, 
with a proportionality constant D, the diffusion coefficient. 
du _ j~.d^y 
dt~ dx^ 
This differential equation is a simple one, and can be inte- 
grated. However, the physical problem need not be solved 
this way. The physical process may be considered directly. 
A molecule at the origin is subject to Brownian motion. 
(This is what makes diffusion work, and is implied in the 
differential equation.) That is, in time, A^, it is going to 
move lS.x either in one direction or the other. In another 
interval of time, A^, it is again going to move either for- 
ward or backward. The molecule describes a random path 
of A^ in each interval of time, A^ and after n steps it will 
arrive at some point x. 

Another particle also follows a random path and ar- 
rives at a different place. If this process is carried out for 
a thousand particles, the distribution of particles at each 
value of .;r is a step function which, in the limit, will be 
exactly the solution of the differential equation. In the 
one-dimensional case the particles distribute themselves 
according to a normal distribution, which spreads out and 
becomes flatter with time. 

These calculations can be carried out very easily on a 
computing machine, because when a particle moves A^i^ 
and A^, it is only necessary to add A^ to its x coordinate, 
with a sign determined by a random number which can 
be fed into the machine. So, fundamentally, this is a very 
straightforward method. It might be crude in complicated 
problems, but one does not have to think very hard to set 
up the problem. 

More difficult problems, which also have a physical 
basis, can be put directly on the machines, by-passing the 
differential equations. For instance, in quantum mechanics 
we are interested in the behavior of chemical systems. By 
a system is meant here an assembly of particles whose 
X, y, z coordinates completely describe the system. Thus, 
if the stationary distribution of these coordinates is known 
the stationary states of the system are known, and if that 
variation with time is known, chemical reactions can be 
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described. If the x, y, z coordinates of each particle are 
used to map out a multi-dimensional space the system can 
be described uniquely by a point in this space. A change 
of the system with time corresponds to a motion of this 
point in the dimensional space. From the point of view of 
the Monte-Carlo Method, the point is a particle. It is an 
abstract particle and does not refer to the electrons or 
nuclei in the system. In quantum mechanics these abstract 
particles do not have to diffuse in the classical sense ; they 
can jump around in the configuration space. These jumps 
are called transitions. In this way a very characteristic fea- 
ture of quantum mechanics can be introduced into the for- 
mulation of the problem. However, to simplify presentation 
and tie the Monte-Carlo Method to what has already been 
discussed at this seminar, we shall take the point of view 
that these transitions are over a small range and are gov- 
erned by a partial differential equation. This is the so-called 
time-dependent Hamiltonian equation: 



i^ 



#(,r,0 
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= Hxl/(x,t) 



with solutions 



■ikt/ft 
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This can be looked upon as a diffusion equation in a space 
with complex coordinates. It has periodic solutions. How- 
ever, by making a transformation to a new time {it/-H) it 
reduces to an ordinary differential equation with real vari- 
ables of the type that Professor Kac^ described : 



du 

ft 



= V'm - V(x)u 



with solutions 



■\t 



u{x,f) = ij/{x) e 

The first two terms correspond to the ordinary diffusion 
equation which I have discussed. The third term is addi- 
tional. The diffusion process corresponds to the same kind 
of random walk, only now when a particle reaches a point, 
X, it is multiplied by a factor ^-^ («')**. Thus, if the potential 
is positive, the particle has less weight when it arrives at 
the region. When V{x) is negative it has more weight. 
Thus, the particles, beside diffusing, actually increase or 
decrease in weight. The diffusion phenomenon governed by 
this type of differential equation, that is by Schrodinger's 
equation, corresponds to the particles diffusing from the 
origin and distributing themselves, at first, in a curve simi- 
lar to the normal one obtained for ordinary diffusion. The 
effect of the potential energy is such that as the particles 
diffuse far out they decrease in weight very quickly, 
whereas the particles diffusing near the origin do not de- 
crease in weight very fast ; so that the quantum mechanical 
distribution function gets cut off at the extremes and dies 
out uniformly throughout the whole region. The distribu- 



tion is, in fact, the actual wave function in terms of the 
coordinates, dying out with a logarithmic decay equal to 
the eigenvalue. The exponential decay of the wave function 
causes some difficulty in accuracy on computing machines. 
To avoid this difficulty a modified potential function can be 
used, 

V'(x) = V(x) - X" 

where A° is an approximation to the lowest eigenvalue. 
With this modified potential function in the exponential 
the total number of particles does not decrease. Thus, the 
particles diffuse out to form an approximation to the wave 
function whose area remains constant and which rapidly 
settles down to the proper value. 

Employing the Monte-Carlo Method in this way, we have 
been able to set up a simple problem, namely that of the 
harmonic oscillator on the 602-A calculating punch, which 
is not an elaborate machine. We have actually been able to 
insert only one card and have computing carry on for a 
whole day. When enough random walks have been made, a 
button is pressed, and the sum of the weight factors divided 
by the number of walks is punched. This is the eigenvalue. 
It is rather interesting to see that, using random numbers 
during the day's calculations, a very definite number has 
been obtained, namely, the eigenvalue of the harmonic os- 
cillator, which is correct at least to a certain number of 
significant figures. If a real source of random numbers was 
available, the process could be repeated on another day in 
which an entirely different set of numbers would have 
passed through the machine, and the same eigenvalue, within 
statistical fluctuations, would be obtained. Thus, the com- 
putations are carried out entirely with random numbers. 
Even in one calculation, the numbers at the beginning and 
the end are entirely independent, statistically. There hap- 
pens to be a sort of Markoff process so that the numbers 
over a period of two or three minutes are related to each 
other, although the numbers over a period of several hours 
are quite independent. 

I want, now, to make a connection of the Monte-Carlo 
Method to the other methods of solving differential equa- 
tions discussed at this seminar. To do this we can discuss 
merely the ordinary diffusion equations and leave out the 
potential terms which is a characteristic of quantum me- 
chanics. The diffusion equation must be written as a differ- 
ence equation : 

u(n,m) = ::<u(n— l,m—l) -\- u(n-\-l,m-\-l)> 

X — nt\x 

t = m[\t 

2/\t = A^' 

which states that the number of particles at x, after one 
step, is equal to one half the probability of the particles 
being either to the left or to the right. The set of difference 
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equations, one for each value of x, form a matrix equation, 
where the distribution along x is represented as a vector : 

U(m) = Tu{m—l) 



T 



The matrix T operates on the original distribution w(0), 
which will be assumed to correspond to particles only at the 
origin and, therefore, has an, element = 1 at a- = 0, and G 
elsewhere. Multiplication of the vector by the matrix T cor- 
responds to the diffusion in a short time. At, the operation 
of the square of this matrix on the original vector to taking 
walks of two steps. The wth power of the matrix corre- 
sponds to taking n steps. Each term in each element of the 
resulting vector corresponds to a single walk, and the grand 
total of all the terms in all the elements of the vector corre- 
sponds to all possible walks. 

The method can be generalized by allowing the particles 
not only to move dx and dt, but to move ndx and ndt where 
the chances of moving 2dx, 3dx and 4dx, etc., are governed 
by some distribution law. It is clear that this distribution 
law is exactly that required to express the second derivative 
in terms of the function to the left and to the right : 

:= 2afcM(^+^A^) k= -Kto-\-K . 



dx^ 



Physically, this corresponds to particles having, instead of 
a mean free path, a distribution of free paths, and the idea 
is introduced that a particle can make a transition from one 
state to another and not merely diffuse over a single inter- 
val, dx. In this case, the matrix T has 2K diagonals, but 
although the expressions for the elements of T m are more 
complicated, it is still true that all possible random walks 
correspond to the wth power of this matrix, so that the situ- 
ation is exactly the same as described in the elementary 
diffusion problem. 

The characteristic vectors of this matrix can be found in 
the usual way of iterating the matrix. Thus, it is seen that 
the Monte- Carlo Method of solving a differential equation, 
when carried to the limit of all possible random walks, be- 
comes the recommended method of finding the character- 
istic vectors of a matrix. It is interesting to see the Monte- 
Carlo Method as a "duz" method, in the sense that it works 



with a 2 by 2 or a 10,000 by 10,000 matrix. The advantage 
of the Monte-Carlo Method is that, instead of computing all 
terms in all elements of the wth power of a matrix, only a 
sample of, say, 1,000 need be taken to obtain results to two 
or three significant figures. It is, therefore, quite clear that 
if all possible random walks were taken, a distribution 
would be obtained which would be exactly defined by the 
iterative method of solving difference equations. 

DISCUSSION 

Dr. Brillouin: There is a problem in connection with all 
these applications of the Monte-Carlo Method that has been 
in my mind for some time, and I would like to ask a ques- 
tion. One of the difficulties in using the machine is that you 
have to repeat the computation a number of times, at least 
twice, to be sure that the machine doesn't make mistakes. 
How can you repeat twice, the same random walk? How 
do you make the checks on the Monte-Carlo Method on the 
machine ? 

Dr. King: As I pointed out, that is fundamentally im- 
possible if you have a real source of random numbers. 
However, the Monte-Carlo Method could be carried out 
very conveniently if there were a hub on every IBM control 
panel that said "random number" on it, and supplying a 
random number. To check the results one would merely 
repeat the whole problem, using entirely different random 
numbers. The eigenvalues obtained should, of course, be 
the same as the first time through, within a statistical fluc- 
tuation depending on the number of random walks taken. 
However, this method does not allow for faulty wiring or 
machine failures. To make sure that no mistakes of this 
type have been made, we have adopted the procedure of re- 
cording the random numbers used with every step. We then 
repeat the whole procedure, using the random numbers in 
reverse order. In other words, we allow the particles to 
walk in the opposite direction from the first case. This usu- 
ally means that the function of the types of random num- 
bers be changed so that a fairly reliable check of machine 
methods has been made. 

Rt^tRtNCtS 

1. Mark Kac, "The Monte Carlo Method and Its Applications," 
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Equation," pp. 89-91. 
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I SHOULD LIKE to be able to make you statisticians 
in one brief exposure, but it seems unwise to try. We are 
going to go over some methods that form sort of a central 
core of the statistical techniques that are used today, trying 
to do it in such a way that when someone comes to you — 
wanting this or that computed — you may have a better un- 
derstanding of why these particular things were chosen. 

By and large, we are not going to discuss the formulas 
that would actually be used in the computation (although 
we shall occasionally refer to those used in hand computa- 
tion). I will leave that to Dr. Monroe^ and Dr. Brandt.^ 
I am going to discuss these methods in terms of how it is 
easiest to think about them. 

My purpose, then, is to supply background: statistical, 
algebraic and perhaps intuitive. 

Interpreting Data 

We shall have more to do with models than you might 
expect — quantitive models for what might have happened. 
This will seem strange at first glance, for most of us usually 
keep this aspect of interpreting numbers in our subcon- 
scious. But the whole of modern statistics, philosophy and 
methods alike, is based on the principle of interpreting what 
did happen in terms of what might have happened. When 
you think the situation over, I think that you will agree. 

There are few problems, indeed, where it is sufficient and 
satisfactory to say, "Well, here are the numbers, and this is 
a sort of summary of them. Now, somebody ought to know 
enough to do something with this !" But many of my friends 
will try to do this, astronomers in war work or sociologists 
studying public opinion ; they try to stop too soon. A rea- 
sonable quantitative model would take them much further. 

In discussing new machines, it has been said that "statis- 
tics is counting." Many think so. But the sort of statistical 
procedures that I am going to discuss are basically proce- 
dures for analyzing measurements, not counts. They can 
also be used for counts which behave like measurements, 
and I believe Dr. Brandt will discuss this. There are other 
ways to bring counts into the picture, but we shall not go 
into them here. 

Simple Models 

When a physicist or engineer thinks of reduction of data, 
his first thought is of a set of points y, one or more for each 



of a set of values x, and a process of fitting a straight line. 
We are going to start two steps below this and work up 
slowly. 

Let us suppose that we have a number of measurements 
of what is supposed to be the same quantity, perhaps the 
velocity of light, perhaps the conversion efficiency of a 
catalytic cracking plant, perhaps the response of guayule to 
a new fertilizer. These measurements will not all be the 
same (if they are, we are not carrying enough significant 
figures ! ) . We must take account of their variability ! 

The simplest way to do this is to suppose that 

1 . they are composed of an invariable part and one that 
is fluctuating, 

2. these parts are united by a simple plus sign, and 

3. the fluctuating part behaves like the mathematician's 
ideal coin, or ideal set of dice, with the contribution 
to each measurement wholly unrelated to that in the 
last, or the next, or any others (nearby or not). 

In terms of this situation, where these parts are things we 
shall never know, we want to get as good a hold on the 
underlying facts as we can — from three values, from thirty, 
or from three hundred. These three suppositions suppose a 
lot, and much experimental technique and much experi- 
menter's knowledge of the subject matter go into making 
them good approximations by doing the experiment appro- 
priately. We shall not go into these problems of design, as 
opposed to analysis, of experiment. 

Of course, not all experiments can be conducted to fit 
such a simple model, and we shall also meet more complex 
ones. These will usually involve more parts, and when these 
parts are connected by plus signs, we shall usually use 
methods which grow naturally out of those used for the 
single sample. 

Notation 

We shall deal mainly with averages and variances. If we 
have, or we think about, N numbers, Wy, w^, . • . , Wn then 

Wj + ^2 -j- . ■ . + Wti 



Wa = average (w) = 
variance (w) = 



.V 



Wi + W|+ . . . + W|r — ]y^(Wi -\- W^ -\- . . . -\- Wn) 
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The variance, as defined here, differs from the mean-square- 
error 6i the physicists by just this denominator of iV — 1 
instead of A'^. There is still a deep schism on whether you 
divide by N or N — 1. 1 am far on the left, and divide every- 
thing of this kind by N — 1 under all possible circumstances. 
It makes the formulas simpler ; I think that is a good reason. 
When you have a set of numbers x-^^, x^, . . . , Xn which we 
think of as a sample from a larger collection, as for example 
in our model just described, we calculate the same things 
but use different words and notation 

•^1 "i~ -^2 ~l~ • • • ~l~ ^n ^■¥ 

n n 



xm = mean (x) 



MSDx = mean square deviation (x) = 

■^1 + ^2 + • • • + Xl—(X^ + -^2 + 



-\- Xr,.)- 



n — 1 
We shall steadily use these three conventions : 

A • in place of a subscript means the mean ( = average 

over the sample). 
A A in place of a subscript means the average (taken 

over the population). 
A + in place of a subscript means the sum over the sample. 
Thus, for example, if yij has been observed for each com- 
bination of «■= 1,2, . . . , c with y= 1,2, . . .., r, then 



3^+3 



.V»3 = -SVlS + ^23 + • • . + 3'C3> = - 

3'2« = -<y2i -\- y22 + ■ ■ ' + y2r}=-y2* • 

This choice of notation and terminology is compact and 
convenient, and we shall use it systematically, but you are 
warned that it is not universal, not general, or, in some as- 
pects, not even widespread. 

It will allow us to state models and indicate possible com- 
putations in a relatively compact and clear way. 

A Single Sampi^e 
The Model 

We are now prepared to take the model for a single 
sample that we have already discussed and express it alge- 
braically. It is 

yi = V + ^i> *'= 1,2, ...n, 

V fixed, ( 1 ) 

ei a sample from a population of size A'^, average ca 

and variance a^. 
that is, we think of the fluctuating parts as a random selec- 
tion of n values from N values. These N values will have 
an average, which we have already agreed to call ca and a 
variance which we now agree to call o-^. We may think of N 
as large as we like, and can easily think of AT = oo as a 
limiting case. We may confine our analysis to finite AT, and 
always take the infinite case in the limiting sense. Any 
practical meaningful new features which might appear in a 
mathematical model for the infinite case would be statisti- 



+ . . . + ^2 



)• 



cally extraneous, and we should have to seek for ways to 
eliminate them by changing the model (this we might do by 
changing the mathematician). 

Illustrations 

Let us imagine ourselves in control of an army camp con- 
taining 50,000 men. We may be interested in their average 
height. Suppose that r] is the average height for the whole 
army of 10,000,000 men, which we do not know ; then we 
can define c as the difference between an individual soldier's 
height and rj. There will be 50,000 such values of e (if we 
admit, for the sake of simplicity, that each soldier has a 
height which can be reliably measured). We may select 200 
men at random (which is an operation requiring care) from 
the personnel files, and have their heights measured. We 
shall have available 

^• = '^ + ^• = 250^^ + ^^ 
and we are interested in 

yA = vy H- CA = average of all 50,000 heights. 
Here N = 50,000, n = 200, and the Greek letters, as usual, 
refer to quantities which we do not know, even after the 
experiment. We are, however, interested in learning as 
much as we can about some of these Greek quantities, or 
about some combinations of them. In this case, we wish to 
infer about ?/ + ca. 

As another case, let us take the measurement of the ve- 
locity of light in a vacuum. Here rj would naturally be the 
"true" velocity of light in a vacuum, if such exists, while 
the e's are defined by difference, as the "errors" of single 
determinations. We have no obvious limit to the size of the 
population of errors ; so we take A^= oo . The average veloc- 
ity measured by this observer, under these conditions, with 
this apparatus is 

yA = rj -{- €A . 

This will not be the "true" velocity because of systematic 
errors in theory, in instrument design, in instrument manu- 
facture, and because of the personal equation of the ob- 
server, to name only a few reasons. These systematic effects 
are reflected in €a. The statistical analysis can tell us noth- 
ing (directly) about ca, since we cannot measure any e, or 
anything related to an c which does not involve -q. We can 
learn about an individual ly-f-e, since this combination is an 
observable value, and hence we can learn statistically, about 

•7 + «A • 
The allowance for ca is a matter for the physicist, although 
the statistician may help a little. 

We should hasten to say that in the models we use here, 
there is much flexibility. Another person might define -q to 
be the average value obtainable by this observer under these 
conditions, with this apparatus. By doing so, he would de- 
fine Ca = 0. Since this would not change the experiment, it is 
very well that we shall find that it would riot change our 
formulas or our conclusions. 
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The Identities — Simplest Case 
We can certainly write that 

yi = y+ (Vi — y) , 

and many college freshmen would like to infer from this that 

yf =3'* + {ji — yY > 

which you know to be wrong unless ji — y* (or y* = 0) ! 
However, the sum of the deviations (yi — y») for all i is 
zero, so that 

Sy!=ty* + t(.yi — yy- 
^nyi + Siyi — y^Y 

^nyl-\-{n — 1)^- (defines j^) (!') 

^f + (^^*-^)' (working form). 
The last line indicates how the two terms are ordinarily 
calculated by hand. It is written in terms of sums instead 
of averages. The divisions are postponed to the last. In gen- 
eral, people who do calculate this expression on hand ma- 
chines makes a regular practice of such postponements. It 
saves miscellaneous rounding errors from arising, and 
makes it clear that certain decimal places are not needed 
after a division. Notice that there is one standard process. 
We shall see again and again that we have summed a cer- 
tain number of entries, squared that sum, and then divided 
the square by the number of entries. This yl/n is a standard 
sort of thing that arises again and again. 

All this has been done as if we really expected y to be 
nearly zero. What if we had expected it to be nearly F? 
There is an entirely analogous set of identities, namely 

tiyi-yy^tiy-yy + t{yi-yy 

^n{y.-Yy + t{y,-y.y (1") 

^ n{ym—Yy -\- {n—\)s'^ (thesamej^!) 

Notice that the term %{,yi—y*y, which appears here, is 



exactly the same term which appeared in the previous iden- 
tity. Thus, it equals (w — l).y^ as before, and can be calcu- 
lated in the same simple way as before. 

When the results are placed in a table, the standard form 
is that of Table Ia, where the mystic letters along the top of 
the table refer to "degrees of freedom," "sums of squares," 
and "mean squares." The entries in Table Ia show how the 
actual entries — the numbers found by computation which 
would be entered in such a table — are related to the actual 
observations. 



Tabi^e; Ia 
Tabular Arrange^m^nt for Model (1) 
Item DP SS MS 



mean 
residue 



1 

n — 1 



n(y%—Y)' 
(n—l)s^ 



i(y.-yy 

s^ 



Average Values under the Model 

The model under which we are working, specified pre 

IN\ 

cisely in ( 1 ) , states that the ( ^ ) kinds of samples of 



size 



n are equally probable. The various quantities we have been 
discussing vary from sample to sample. But we can deter- 

( N\ 

mine their average values — averaged over the I « ) kinds 

of samples of size n — in terms of -q and ca and a^. Together 
with some variances, these are given in Table Ib, page 98. 
The essential things to notice are the average values in 
the two bottom rows of Table Ib ; the average values of the 
two mean squares in Table Ia. The average value of the 
"mean square for residue" is just a^, the variance of 
the fluctuating contribution. The average value of the 
"mean square for the mean" consists of two terms, 

(1 — "at )t''^> which is nearly a^ when the population is large 

(and is otherwise smaller), and n{yt, — Yy which is zero 
when the contemplated value Y is equal to the population 
average y*, = -q -\- t^. 

The first essential point to be gained from these average 
values of mean squares ( which from now on we shall sim- 
ply call "average mean square" and abbreviate by "AMS") 
is this. These average values really say that : 

1. All the systematic contribution has been siphoned off 
into the mean square for the mean. 

2. As much of the fluctuating contribution as possible 
has been siphoned into the mean square for residue. 

3. We know how much, on the average, of the effect of 
the fluctuating contribution remains in the mean 
square for the mean ; we know this in terms of o-^ and 
so can judge its size from the mean square for residue. 

This is the qualitative picture — one you need to understand ! 

Interpretation of Anova Table 

Table Ia is called an analysis of variance table (which I 
often like to abbreviate to "anova table"). How do we in- 
terpret specific tables containing numbers? We know that 
we must face sampling fluctuations from sample to sample, 
but if we neglect them momentarily, we can learn much. 

The average value of the residue mean square is o-^, bare 
and unadorned; it tells us about <t^. The average value of 
the mean square for the mean is complex ; it has two terms. 
We can avoid this complexity by forming 

l{(n,ean square for .he „,ean)-(l - 0-;- X" "") 

whose average value is 

i{»(r-,.)'+(l+|p).'-(l-f).j= iY-y.y . 

On the average this component of mean square for the 
mean tells us about (Y — yt.y. Thus, an observed large 
value suggests that Y — y^. is not zero, while an observed 
value which is small, zero, or negative indicates that Y — 3;^ 
could be zero (for all this sample tells us) and that Y — y^. is 
surely small. 

By analogy, we define the CMS (component of mean 
square) for the residue by 

CMS residue = MS residue. 
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Tablk Ib 
Average VaIvUES and Variances eor Modei. (1) 

Quantity Average Value Variance 



V 




V 







Ci 




Ca 




(■4)- 


3' 




1/ + tA 




(-*)•■ 


y* 




1/ + CA 




il-i)- 


y. - Y 




V + e^-Y 




t"ir)"' 


(y-yy 


(tj + eA 


-^)^+(^ 


"ir)" 


2 


t(yi-yy 




{n-l)a^ 




. . . 



tiy-yy == n{y.-Yy n{r, + eA-Yy-\-(^l-^^ 
1 



n — 1 



t(yi--yy 



since its average value is just ct^, and notice that we may 
write 



CMS mean = - MS mean 
n 



\n N) 



MS residue. 



Now a very usual case \s N= oo. Let us suppose that this 

is so, and that we find 

MS mean = 1000 , 

MS residue = 10 . 

We must conclude, if there were at least three observations, 

that the CMS mean, which came out to be 

900 

n ' 

is quite sure not to be zero on the average. Thus, Y — y^. is 

not zero, and we do not believe that Y can equal y^.. 

On the other hand, if 

MS mean =9, 10 or 11 

MS residue = 10 , 

we have . . 

CMS mean = --,0,- ' 
n n 

and we must conclude that its average value might be zero. 
Thus, Y — yA might be zero, and Y might be the unknown 
yA. Moreover, Y — yA is probably small. 

Thus, in extreme cases a look at the mean squares may 
tell us whether Y is quite sure not to be equal to (or near 
to) yA or whether it may equal ^a (and is quite surely near 
it). In intermediate cases we should have to think hard, or 
use a carefully worked out procedure (such as we men- 
tion shortly). 



Notice that the average values of the mean squares and, 
at least by implication, the components of mean square play 
an essential role in drawing such conclusions. 

Test Ratios and Confidence Intervals 

Now you may wish to make a critical test of the null 
hypothesis that some Y you have selected may be equal to 
^A. This is ordinarily done with one of the two ratios indi- 
cated in Table Ic. 



TabIvE Ic 

Test ILvtios eor Modei, ( 1 ) 

^_ njy.-Yy _ (y.-Yy ( y.-Y y=: f 



1 



t 



n 

y-Y 



p,{yi-yy 



1 o 

- S" 

n 



^/s'/ni 



yjs^/n 

If the critical value of f = ta, then confidence limits for 
Va = iy + €A 
are given by 



P is used frequently for what is called a variance ratio. In 
this case, as usual, it is just the ratio of the two entries in 
the mean square column. In this particular case (as always 
when the numerator has only one degree of freedom) it is 
exactly the square of the very simple ratio denoted by t. 
(This is Student's t.) 
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If a given value of Y gives rise to a value oi t {or P) 
near zero, we must, so far as this one set of observations is 
concerned, admit that 3;^ (which we don't know) might 
equal this given (or contemplated) value Y. If some Y is 
far enough from the observed mean y; then t (and also 
P=t^) will be large, and we shall be more or less sure that 
this Y differs from y^. Those values of Y, which might 
reasonably be the unknown value of yt, form a confidence 
interval for 3;^. The last line of Table Ic shows how easy it 
is to compute such an interval. The interpretation of the 
interval is merely "yt, is likely to lie in here," with the 
strength of the conclusion depending on ta and getting 
stronger as the interval increases in length. 

The Diagram 

Figure 1 is a very simple diagram of the sort that we 
can use again and again in more complex cases ; at least it 
helps me in understanding what is going on. The vertical 
increments are sums of squares, while the horizontal incre- 
ments are corresponding degrees of freedom. Thus, the 
slopes are mean squares. We obtain one slanting segment, 
which we shall call a trace, for each line in the analysis of 
variance table. 

The possible allowance for fluctuations to be made in the 
sum of squares for mean is indicated by the small shaded 
triangle, whose altitude is s^. In the right-hand portion of 
the figure, this possible allowance is divided into the ratio 
of A'' — « to n to produce the actual allowance {\—n/N)s-. 
This portion of the figure shows how the failure of Y to 
equal the Ja. seems to contribute to the total sum of squares 
of deviations from Y. 



Apparent Partition 
of Variation 




— 1~\ H 



n-1 I 1 

Degrees of Freedom 




Apparent Partition 
of Variation 



!bH 



1 



n-1 I 1 

Degrees of Freedom 



Figure; 1. Anai^ysis of Variance: Diagram 
FOR Model (1), Ge;ne;rai, 



FiGUR:e 2. Analysis of Variance Diagram 

for Model (1), Special Case 

(« = 00 and Y near y^) 

If A^= 00, and the sum of squares for the mean came to 
exactly this height (as it does not in Figure 1, but does in 
Figure 2), the component of mean square would be zero, 
and we should have no reason to believe that Va should be 
different from Y. Thus, the fact that the mean trace is 
steeper than the residue trace indicates some evidence that 
Y is not equal to y^.. 

To put it another way, when Y does happen to equal y^ 
the slopes of the trace for the mean and the trace for the 
residue both average to o-^. Thus, their failure to agree in 
slope — nicely measured when N = co by the ratio, F, of the 
slopes — is an indication, not a proof, that Y does not hap- 
pen to equal ^a. 

The reason that we must use the ratio and not the dif- 
ference is easy to see. The vertical scale of this diagram 
depends on the units in which we express our measure- 
ments. [If we change from feet to inches, «(y» — YY in- 
creases by a factor of 144!] So differences in slope would 
depend on the units used, which would clearly be wrong. 
The ratio does not depend on the units. 

What sort of ratio of slopes you allow, before you feel 
that the Y contemplated cannot reasonably equal y^., de- 
pends on n, among other things. If m = 1,000,000, so that 
there are 999,999 degrees of freedom for the residue, then 
a ratio of 5 is suspect. But, when « = 2 so that there is only 
1 degree of freedom in the residue, a ratio of 150 may well 
escape suspicion. In any event, the diagram places the situ- 
ation before you honestly and fully. 

Regression 

We have gone further into the simplest case than we 
intend to go into the others. We shall take the next group 
of cases, which include fitting straight lines, curves and 
surfaces (in their simple cases), and jnultiple regression 
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analyses (sometimes miscalled correlation analyses) more 
rapidly. Here these are lumped together under the heading 
of regression, and can be treated together. We shall begin 
at the beginning, and build uj) to the general case. 

Ultrasimple Regression 

We begin with the simplest case of all, where the model is 
'ji = i8^i + H, ? = 1,2, . . ., w, 

^ fixed, 
^ .«-i |- known without error, (2) 

■{ ci }■ a randomly arranged sample from a population 
of size N, average zero, variance o-^. 
The assumption "known without error" means, actually, 
that you have measured x so much better than y that you 
don't have to consider the errors in x. We have assumed 
CA = because we really believe the "true" line 

3' = ^''^' 
passes through the origin. 
The identities are 

^h^tx^-\- {n—\)s' (defines .y^ (2') 



where 
and 

'tiyi- 



t^l 



+ \tfi - ^^^} ' (working form) 



b = 






-Bx.y ^tihx—BxiY + ^{y-bxiy- (2") 

^{h—By^x\-\-{n—\)s' (same.?-) 



__ {^Xiyi—B^x^'^^ 



fUy; 



(X-^iyQ 



■K 



( working form). 
Notice that the y\/n term is back in its more general 
form 

[^(coefficient) 3;i]V^( coefficient)^ . 
In treating the single sample all the coefficients were 1; here 
they differ from one i to another, being just Xi. It's the same 
principle in action. 

The analysis of variance table, including a column for 
the average mean squares, is given in Table IIa. This table 



explains how DF, SS and MS and CMS are related to the 
observations, and how the average mean squares and com- 
ponents of mean square are related to the population. 

Notice that w in 1 — (n/N) has been replaced by 
x'i^'^x^y^, which reduces to n when the Xi are all equal. 

An actual table would have numerical entries through the 
MS column or the CMS column and might or might not 
have some or all of the algebraic entries in the AMS and 
ACMS columns to guide the reader. 

We can test the null hypothesis f3 = B, and set confidence 
limits for y8 just as we did with the mean. The formulas are 
set out in Table IIb. 



Tabi^e; IIb 
Te;st Ratios and Conf'ideince Limits i^or Mod^i. (2) 



s^ 






If the critical value of f — ta, then confidence limits for ^ 
are 



b±fa 



it^iy'' 



Thus, we can examine any contemplated value, B, for the 
slope (which is really /?) and see if the observations object 
to it vigorously enough. The confidence limits will surround 
those values not objected to by the observations. The dia- 
gram for this case looks just like the one for the mean, and 
we shall omit it. 

Casting Back 

Suppose we take the special case where all Xi are the 
same — in fact, all equal to A. The model becomes 
yi = ^k -\- ei, i = 1,2, . . . , n, 

and we see that we have a special case of model ( 1 ) where 
rj = ^X. It is a special case only because we assumed ca = 



Item 



TabIvEJIIa 

Anai^ysis of* Variance^ Tabi^i; I'or Mode^i, (2) 

DP SS MS CMS 



-{'- 



slope 1 {b-By^xl {b-By%xi (b-B)'- 

residue n — 1 (w — 1).?^ s^ s^ 

AMS ACMS 

(B-^y 

residue a^ a^ 



Ntx'iJtxf 



slope {B-^y tx\ + (l - ^) a^ 
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Tabi.^ IIIa 




Anai^ysis o-e Variance Table eor Modei. (3) 


Item 


DP SS 


MS CMS 


mean 


1 n(y.-Y,y 


n{y.-Y.y {y.-Y.y-{^-^s-^ 


slope 


1 {b-Byx{xi-x.y {b- 


-Byt{x,-x.y {h-By - s^t{^i-x.y\-' 


residue 


n — 2 (« — 2)s^ 


s-" s'~ 




AMS 


ACMS 


mean 


n{y.-Y.y-[\-j^y 


(y^-Y.y 


slope 


{B-^yt{xi-x.y + a' 


(B-^y 


residue 


a' 


a^ 



in model (2). But we can always force ca = in the first 
model by defining rj properly. Thus, we see that fitting a 
mean is a special case of fitting a slope. This may surprise 
us a little, but when we think a while it becomes reasonable. 
We shall see more and more of this sort of thing. 

The Linear Case 

We are now going to branch out boldly, and let the line 
go anywhere — no longer must it go through the origin. The 
model is 

yi= (x-\- ^Xi -f ci, i = 1, 2, . . . , n 

■{xi\- known without error ( 3 ) 

■{ ci }- a sample from a population of size N, average 
and variance o-^. 
This is appropriate when we wish to fit a line to observa- 
tions of X and 3; where 

1. the errors and fluctuations in x are negligible com- 
pared to those in y, and 

2. the size of the errors in y do not depend on whether 
jr or 3; is large or small. 



b = 



There are other procedures for more complicated cases, 
and for those you may start with references 3 to 11. 
The identities are, briefly, 

tyi ^ty+ t\b(x~x.) p + ti (y—y) - b{x~x.) p 

(3') 
^nyX — &^ ^{^i—^^y + (w — 2).$-^ (defines .y^) 
where 

t{^i—x»){yi—y») 
^{xi-x.y 

and if Yi = A -{- Bxi is the contemplated line 

tiy—YiY = niy.-Y.y + {b-By%{x~x.y 

-f {n-2)s- . (3") 

The tabular arrangement is given in Table IIIa, and the 
test ratios and confidence limits are those of Table IIb 
where y^ is defined by 

yA = ce-\- ^Xm 4- €a 

to be just the average y for the fixed set of .ar- values that 
we have observed. 



Table IIIb 
Test Ratios and Confidence Limits eor Modee (3) 



P 



P = 



n(y—Ym)'' 
s- 



{b-Byt{xi-x.y { b-B 



= y => = r , (for mean). 

\\/syn) 



^ = [ ^ ^ V= f , (for slope). 



If the critical value of t = ta, then confidence limits for yt, are 



and for y8 are 



y ± ^a-7= 



b = ta 



vt(xi—x»y 
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Apparent Partition 
of Variation 







n~2 I 1 

Degrees of Freedom 



Figure; 3. Anai^ysis of Variance Diagram 
FOR Mode;i. (3) 



We have broken up the sum of squares in a way entirely 
analogous to the one we used in the simple cases. The dia- 
gram is shown in Figure 3. Just as before, the test ratios 
(Table IIIb, page 101) are ratios of slopes of traces in this 
diagram. The shaded portions show how much allowance 
might carelessly (triangles) he made and is actually (blocks) 
made for the contribution of fluctuations to the sum of 
squares for the mean and for slope. Just as before, these 
allowances are correct on the average. The actual numerical 
values in the diagram correspond to a J5 which seems likely 
not to = /3 and to a ¥• = A -\- Bxm which is quite clearly 
not = yA = cc -\- fix: 

It may be interesting to write down the working forms 
of the three sums of squares. They are 

{y,- {nA^Bx,)Y 
n 



tA 



1 . 

-^; 
n 






Orthogonality 
Without explanation, the last model was taken apart into 

1. a piece for the mean a -)- Z^-*"*? 3^^<i 

2. a piece for the slope ^. 



It would be natural for you to ask why we didn't take out a 
piece for the intercept a and a piece for the slope fi. The 
answer would be "this would be inconvenient, because the 
natural estimates of ot and ^ are not orthogonal." And so 
you would wonder — what is orthogonality? Let us try to 
answer this question. 

First, the change to ^a = a + fixm and p really corre- 
sponds to writing 

a -\- pXi = {(x—pxm)\ -f p{xi—xm) . 

It corresponds to using 1 and (xi — x») as the quantities 
whose coefficients are to be found. The orthogonality of 
these quantities is easy to express algebraically. The con- 
dition is 

X l(xi—X9) = , 

which we know to be true. But why should we be interested 
in these quantities ? 

Let us write y» and b in terms of b, y8 and the ti. We have 

y»=<X -\- ^Xm -{- €m = yA -\- em — eA 
i'X(Xi—X9)^=Xi<X + ^Xi -\- H— («+/8^»-f-c«) }■ (Xi—Xm) 

=fii(Xi—Xmy + X(ci— £•) (Xi—Xm) , 

=^t{''^i—X»)^ — t(Xi—Xm)ei 
SO that 



y 



■Ya = £• — Ca = — X 1 i^i — ^^) 



b—/3 = 



1 



XiXi—Xm) 
1 



2t(Xi — Xm)€i 



iS(^i— ^•)(£i— £a) 



XiXi—X»)' 

We are going to inquire about the tendency of these quan- 
tities to fluctuate together. To do this, we begin with 

average i (ti-eAY} = ( ^ ~ jy") *^^ ' 

average ] (ci— ca) (cy— «a) }■ = 






hence 



average ^ (y*— ^a) (b—^) } 
1 



-n2(..-..)^^<''Slte— )+0} 
which vanishes because 

tl(Xi—Xm) 

does. 

In general, the condition that 

average ■{ ^a^ ( ci — ca ) X^i ( «i— ^^ ) l' ^ ^ 
is that 

^ai&i = . 
Thus, one meaning of orthogonality is "no tendency to 
fluctuate together" (as measured by this particular sort of 
average). Since it is clearly convenient, but not essential, 
to work with quantities which do not tend to fluctuate to- 
gether, this meaning will perhaps content you now. An- 
other meaning is discussed on page 104. 
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Multiple Regression 

Now we can look at the machinery for what is called 
multiple regression — a very general procedure. The model 
looks as follows : 

i = \,2, . . . ,n 
^ \xji\ is known without error (4) 

{^i} is a randomly arranged sample from a popu- 
lation of size A^, average zero and variance a^. 
The independent variables, or carriers of regression, x^, x.^, 
. . . , Xm. can be anything. They can be related or unrelated, 
constant or not. Thus, the case 

w = 2: x^^l, x,y^ X , 
covers the general straight line, while 

w = 3 : .^1 ^ 1, x^^u, x^^v , 
covers fitting the general plane to the values of 3; in terms 
of (l,M,t/). In a medical case, for example, we might take 

w = 5 : x^^\, X2 ^ log blood pressure, x^ ^ age, 

x^ ^ weight as per cent of normal for height and age, 

x^ ^ excitability. 
You can propose all sorts of examples in the fields that in- 
terest you. 

There are statisticians who prefer to deal with the origi- 
nal variables x^, x^, . . . ,Xm, all together and on an equal 
footing. This is permissible, but I don't like to do it, since 
it allows no place for using the extra knowledge that you 
almost always have. It also leads to matrix formulas that 
are too much trouble to remember for one like me who does 
such calculations infrequently. (If you do such computa- 
tions regularly, and under circumstances where you are 
sure that you want complete analyses, then by all means 
use the matrix formulas.) Instead, I am going to describe 
a one-at-a-time procedure, where we start with the most 
promising variable first, the next most promising variable 
second, and so on. (We suppose the numbering changed 
so that x-^ is most promising, x^ next, and so on. ) Then we 
form a new set of variables 



^1. 
X2, 



— x.^ 

— x._ 






'32'1 '^2'1 



which are orthogonal (the dots do not mean averages here) . 
This is entirely like changing from 1, .r to 1, {x — x*) in 
fitting a straight line. The consequence is a simple proce- 
dure for fitting the constants in 

^l-^l ~r C'2'l'*'2'l I '^3«12-'^3'12 I • • • 5 

a procedure of fitting them one at a time from left to right. 
Thus, we take out a piece for x^ first (this is the biggest if 
we guessed well) then one for x^.^ (this ought to be rela- 
tively big, too) and so on. 

Notation and Identities 

To make the wheels turn smoothly, we need a complex- 
seeming but simple operating notation for certain sums and 



the results of simple operations on them. We write 

[JM = Si xjiXki , 1 ^j,k^m , 

[yj] = ti yi^ji > 1 ^ ; ^ w , 

and then define dotted quantities according to the general 
symbolic principle 

[j • L,m-L] [k-L,m'L] 



[j-Lm,k-Lm] =[j-L,k'L] — 
Thus, we have 

[;. 1,^-1] = [j,k] 



[m-L, m-L] 

[y>i][^-i] 



[/•12,^-12] =[j-l,k-l] 



[1,1] ' 

[i-l,2-l][fe>l,2-11 
[2.1,2-1] 
(and so on) , 

[ri,y-i] = M-™^. 

(and so on) . 
We also define the new variables by 

^-^ X -x.-h-x 



_ [y-i,2-i] 

•*->i2 - ^i-i [2- 1,2-1] 

(and so on) , 



— ^j'\ ^J2'l ■^2«1 > 



and the residuals by 



y-i = y 



[yX 



^1 = y — ^1^1 



3'o2 — y-1 rO 1 oil "^2'! — V-l ^2«l ^2-1 



[1,1] 

[yi,2-il 

[2-1,2-1] 
(and so on) . 
With all this machinery chosen for our convenience, it 
would be strange if the identities were not relatively simple. 
They are 

^ 2 [y,ir , [yl,2-l]^ , [yl2,3-12]^ , /. . x 
^^--[Ij] + [2-1,2-1] + [3-12,3-12] + ^^° ^ ^^™^) 

+ t(y-i2s...my (4') 

= [l,l]P,+ [2-l,2-l](&3.x)^ 

+ [3-12,3-12](&3.,2)^ + ...-F (n-m)s' 
and 

Xiyi — (^i^i+-S2.i^2. 1+^3.12-^3.12 + (to m terms)) p 

^[1,1] (6 -5,) + [2-l,2-l](&2.i-5,.0^ (4") 
-|- (tow terms) -f- (m — m)s^. 

The working computations would go in terms of the 
brackets and the first form of (4'), and notation would be 
simplified by labeling [j'L, k-L] just [jk-L]. 

Table and Diagram 

The results would be given as in Table IVa, which fol- 
lows the pattern of the previous analysis of variance tables. 
Usually such a table is given for B-^ =^2-1 = • • • = 0, but 
this is habit rather than thought-out policy. Such a table 
should always be supplemented by a table of values of 
bi, &2-1, ^3.12, • • • ^^^ usually by a table of the expressions 
of .^a.i, ^3.12, and so on, in terms of the original variables 



•^1, -^2' 



, Xpi. 



104 



COMPUTATION 



Item 



Tabids IVa 
Analysis of Variance; for Modfi, (4) 
DP SS MS 



X, 1 [i,l](&_5J^ {\,\]{h-B,y 

X,., 1 [2-\,2-\]{h,.-B,.,y [2-\,2-mh.^-B,.,Y 

V,.,, 1 [3-12,3-12](&3.,,-53.,3)^ [3-12,3-12](&3.,,-53o,) = 



residue n — m 



CMS 



AMS 



X. (6.-B.)>-(l-^)j^ [U](A-B.)'+(l-^)^' 



^2-1 (^2.1— ^2-l)'- 

'''3'12 (^3'12 -^3'12/ 



[2-l,2-l] 
[3- 12, 3- 12] 



[2-l,2.1](/?,.,-5,.,)^ + cr^ 
[3-12,3-12](;83.,-53.,,)^ + cr^ 



residue 






(^3-12 — -^3-12)^ 



residue 



The corresponding diagram follows along the usual pat- 
tern and is shown in Figure 4. This figure is drawn for the 
special case 

w = 4: jTi ^ 1, x^^x, Xc^^x^, x^^x^ . 
Thus, the various components, those for 1, x — x; x^ — 
. . . , x^ — • • • , are called "mean," "slope," "curvature," 
and "twist." 

Geometric Interpretation 

All these mysterious notations and identities have a nice 
geometric interpretation. Just make a vector out of each of 
the m variables — m vectors in an ^-dimensional space. The 
initial set of coordinates in the space is like the statistician's 
sample space ; the coordinates of J in this system are the n 



observed values of x^ for the n observations, and so on 

X^ = i-*"lif 
■^2 ~~ 1 •^21 f 

y = ivi}^ • 

Now, experimenters and quantities of interest being as they 
are, the m vectors i\, j?2 > • • • >^m that correspond to the m 
variables are unlikely to be at right angles to each other. 

But the aim of our fitting process comes out to be just 
finding the component of 'y in the w-dimensional space de- 
termined by ^1, i^2J • • ■ > ^m- This would be easy if "x-^, . . . , x^ 
were at right angles. So we set out to force them to right 
angles and calculate the projection all at once. 
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Apparent Partition 
of Variation 




Degrees of Freedom 

Figure 4. Anai^ysis of Variance: Diagram 

for modei, (4) 

Spe;ciai, Cask of Polynomial Fitting 



It is easy to replace 'x^, . . . , ^m, 'y by their components 
perpendicular to Xj^. We have only to calculate a few dot 
products and proceed as follows : 



Xn ,1 Ji « 



Xri 



(^1 • ^1) 



I, Xm ' -^ 1 ) 

(y -^i) - 



Then we can shift from ^g.^, . . , , Xm.i, 3^i to their compo- 
nents perpendicular to ^2-1 • These new vectors will still be 
perpendicular to 3?i. And so on. 

By comparison with the operations on brackets specified 
above, you can see that this orthogonalization procedure is 
just what we have been doing. From the vectors Xj^,X2,..., 
Xm, we have constructed new vectors x^, x^.-i, J^g.ia , . . . , at 
right angles to each other, and we have found the compo- 
nents b-^Xi, &2.1 X2.1, . . . , oi y along the new vectors and 
the residual vector 3^12 ...m which is the component of y 
perpendicular to all the ^'s. 

All this still may seem complicated. So let us go back and 
fit a line to two points. We take « = w = 2, jtr^ ^ 1, 
^2^ X and assume the observations 
(x = 2,y=:l) 
(;r = 4, y = —2) 



The vectors are shown in the original coordinate system 
in Figure 5. 
The result of passing to ^g-i and y.^ is to replace (2,4) by 



— 1, -|-1) and (1,-2) by( 1-, — 1:= j. This is done through 



— » — » A— > 

3'.i =y — Y^ • 



Now we notice that 



•2-^2.1 



and hence 



1 



y - 2 '^1 2 ^'"' 



4x^ 



Recalling that Xj = 1, Xg = x, we see that we have fitted 
the line 

y = 4 — 2^ 

to the points (2,1) and (4, — 2). It is easy to see that this 
is a correct fit. 

The geometric interpretation is the same for any regres- 
sion problem. It merely requires more than two dimensions 
for the picture. 

If we had not orthogonalized, then the problem of finding 
the projection of y^on the w-dimensional plane of jTi, J?2» • • • > 
Xm leads at once to m- equations in m unknowns. As prac- 
tical computers, how would we have solved these equations ? 
By some method of elimination — whether we talk of Doo- 
little, Crout, or Dwyer's square root method. Geometrically 
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this means by orthogonalization, by something close to our 
actual procedure. What difference might there be ! 

Really the difference is only this : we have not abbreviated 
the method, we have put down all the steps. This leaves us 
with a chance to change our mind — if [7,123456, 7.123456] 
is very small, so small that the errors of measurement we 
have been neglecting account for most of it, we can drop 
variable 7 without loss of work and go on to 8.123456, and 
then to 9.1234568. We have written or typed or magnetized 
or punched or mercuried some extra numbers. With these 
we have bought insight at intermediate points and flexi- 
bility. I like to do it this way ; you may like to do it another. 

References 

Now there are more complex problems in regression, and 
more complex ways to do simple problems. These we can 
only cover by reference. 

For problems where the observations are of varying ac- 
curacy or where the coefficients appear nonlinearly, the 
classical methods are set out in reference 3. 

For problems involving polynomials at equally spaced 
intervals, much time and trouble is saved by the use of al- 
ready prepared orthogonal polynomials. These are available 
as follows : 

To 5th degree, to 52 points, in reference 4. 
To 5th degree, to 75 points, in reference 5. 
To 5th degree, 53 to 104 points, in reference 6. 
To 9th degree, to 52 or more points, in reference 7. 
To 5th degree, to 21 points, in reference 8. 

For problems involving error in more than one variable, 
the user should read references 9 and 10, and for further 
study the references given there. A very condensed sum- 
mary of many theoretical results may be found in refer- 
ence 11. 

Application of regression ideas to more general problems 
can be found in reference 12, and computational suggestions 
can be found in some of the texts in references 14-18. 

SiNGIvE; ClyASSlFlCATlON 

Several Groups — A Single Classification 

Regression, in the sense that we have used it — curve- 
fitting, general mopping up with sums of terms, and the 
like — accounts for many physical and chemical applications. 
But, in many fields the type of analysis that we now enter 
is the standard. Particularly in agriculture, only slightly 
less in engineering and applied sciences, and to some extent 
everywhere, the comparative experiment is king. 

Simple before-and-after experiments, or comparisons of 
two brands, two processes, two finishes or two raw mate- 
rials, are easy ; by taking differences you return to the type 
of case we have discussed. We need to consider compari- 
sons of several categories. The simplest experiment is one 
in which you have m^ observations in category j, where i 
runs from 1 to c. That is, «x observations on the first brand, 
the product of the first process, material covered with the 



first finish or units made from the first raw material ; «2 ob- 
servations on the second brand, the product of the second 
process, material covered with the second finish or units 
made from the second raw material ; and so made on all 
categories. The model will exhibit each observation as the 
sum of a contribution depending on the category and a fluc- 
tuating contribution. 

Here you can use quite complicated models for the fluctu- 
ating component with good sense and good results. For 
many of these models the part of the analysis which we are 
going to describe is the same. So I am not going to say just 
what I assume here. If you assume, for example, that all 
the fluctuations for all the categories come randomly out of 
one big population of fluctuations, then you will have a 
model that will fit a lot of circumstances. Everything that 
is going to be said here will apply to that model. If you 
want more ideas about possible models, read pages 69-75 of 
reference 13. 

We need some sort of a model, however, so that we can 
describe average values and variances of things. We specify 
a simple one, namely, 

yij = A -f r/i + €ij, 1 ^ y ^ Wi, 

A fixed (5) 

-JT/i}- a sample from a population (of categories) 

of size M, average t/a and variance o-^. 
]eij\- a sample from a population (of fluctua- 
tions) of size N, average €aa, and variance o^. 
This is a good standard model, but not the most general for 
which our analysis is suitable. 

The categories in such a comparative experiment may be 
anything. In a study of screws, they might be different 
automatic screw machines, where you had taken a handful 
of screws from each for measurement. Or they might be 
the different times at which you had taken a handful from 
one machine. Or they might be different months in which 
you had sampled the whole factory's production. In agricul- 
ture they may be different varieties receiving different fer- 
tilizers. The categories may be individual operators of a 
chemical unit process, or they may be different fatigue 
states (as measured by time on shift) of a single operator. 
You have a choice of a lot of things here. 

Identities 

Our identities follow the standard pattern ; here there are 
three pieces, as we see in (5') and (5"). 

^yl-^^yi. + tiyi.-y..y + t(yv—yi*y (5') 

^ny9» -\- Xi ^iiyi^—y)'^ + {n—c)s^ (defines s'^) 

tiyi-yiY ^tiyi^-y^y -t(yi~yi*r 

^ti^i(yi*-yiy+ (n-c)s' (5") 

^n{y»»~Y»)^ -\- 

ti^iiyi^—y—^i+y*)^ + (n-c)s\ 
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Where the dottings in place of the i's, mean weighted aver- 
aging, that is, 



>• = 



F. = 



2. 



n = ^fii ' ' n = ^Wi 

and n is defined as in these denominators. 

In terms of the second line of (5') the three pieces stand 
out clearly. 

1. A piece depending on ymm which expresses the fact 
that the sample grand mean is not zero. 
A piece comparing the category means among them- 
selves. 
3. A piece expressing fluctuations within a category. 
Just as in our first case, we have siphoned into the last 
term all that we possibly could of each of the fluctuations 
without getting category or grand mean effects. Likewise, 
we have siphoned as much as we can of the category-to- 
category differences into the second piece without getting 
grand mean effects. Our purification is only partial, but it 
is the best that we can do. It is the old method, applied at 
two levels instead of one, in two stages instead of one. We 
have isolated the fluctuations within categories from the 
category means as well as possible. Then we have isolated 
the differences in category means from the grand mean as 
well as we are able. On two levels at once, we use the same 
process which you use unconsciously when you take an 
average. 

Tables 

The elementary question that is going to be asked is: 
"Are these categories different?" This is only the first 
question, and those who stop with it are probably not getting 
what they should out of the observations. From the stand- 
point of the computing group, it doesn't make much differ- 



ence, because if they have answered this they have, ready at 
hand, the other numbers which might be needed. Whether 
asked for or not, always send the category means back up- 
stairs with the analysis of variance table. Don't let the 
statisticians forget the means for the sake of significance 
tests ! 

The form of the analysis of variance table is shown in 
Table Va. We have shown the one degree of freedom for 
the mean which many leave out. It has nothing to do with 
the comparison of categories, and since that is what such 
analyses are usually for, it is often omitted. But there are 
analyses that come in exactly this form where this line con- 
tains key information. If you attacked the problem which 
was mentioned previously at this Seminar — getting the 
average height of all the men in the United States — it would 
not be very practical to try to draw a random sample di- 
rectly of a thousand men out of all the inhabitants of the 
United States. No one has a convenient card file that you 
dan enter with random numbers and pull out names. You 
would want at the very least to break up the United States 
into pieces, and select randomly and measure two or three 
men in each of several randomly selected pieces. If you did 
this you would have a situation that comes under this 
model ; because, if you broke up the United States into 
pieces in any reasonable way, the average heights of the 
men in the different pieces would be different, and these 
differences from piece to piece might be crucial in fixing the 
accuracy of your over-all mean. 

There are approximately 3,0(X) counties in the United 
States. Some of them, like Manhattan, are a little large and 
inhomogeneous. Let us think in terms of 10,000 categories. 
These are to be geographical regions, each with about the 
same number of men. (What is a man, anyway?) If we 



Tabids Va 




AnaIvYSis 01*' Variance TabIvD i^or Modei. ( 


:5) 


Item DP SS MS 


CMS 


mean 1 nymi nymi 
categories c— 1 :SiWi(3'i«— >•)" ^Zj ^i«i(3'i*- 


(*) 


-yy (*) 


within n — c {n — c)j^ s^ 


s^ 


AMS 


ACMS 


mean (l - ;) .^ + (l _ ;^;^) .| + ....^ 


yd 


categories ^'+^(^_i) ^^ 
within CT^ 


2 

a" 


*Best computed from the numerical values of the coefficients in the 
AMS column. 
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selected 100 of these at random, and then selected three 
men for measurement randomly within each of the hundred, 
the grand average tells us a lot about the average height of 
U.S. men. The grand average is going to fluctuate for two 
reasons. One reason is that if you repeat the process you 
would not have the same three men in a given category. 
The other is likely to be more important ; if you repeated 
the process you would have a different set of 100 categories. 

You have here a situation where it makes sense to write 
for any individual 

height = U.S. average + (category average — U.S. average) 

-}- (individual height — category average) . 
Now, our grand average is the sum of the grand averages 
of the three contributions for each individual. If you redo 
the whole process, and use a new sample of 100 categories, 
then the average of the 100 category averages will be dif- 
ferent ; the grand average of the second contributions will 
be diflferent. We must allow for this as well as for the fluc- 
tuations in the grand average of the third contribution. 

Before we go on, we notice that Table Va is a little com- 
plicated, and conjecture that this is due to the possibility of 
having different numbers of observations in the different 
categories. So we treat the case (Table VI a) 

like (5) except ni^riori= 1, 2, ... , c. (6) 
Here things are quite simple in every line, except that for 
the mean. 



Apparent Partition 
of Variation 




n-C I C-1 

Degrees of Freedom 



FiGURU 6. Analysis of Variance Diagram 
FOR MODE^L (6) 



Diagram 

Having the tables, we can now set forth the diagrams, 
which we do in Figure 6 for model (6) . 

If we examine this diagram we see that it is much like 
the other diagrams. We have the traces, one for each line 
of the table. Clearly, one degree of freedom goes into the 



grand mean, and that is lost from among the c categories. 
So there must be c — 1 degrees of freedom for categories. 
This disposes of a total of c degrees of freedom ; there were 
n observations. Take c from n and you have n — c, the num- 
ber of degrees of freedom within categories. 

We should certainly call the differences between cate- 
gories "apparently negligible" if the traces for "within" and 



Item 



DP 



Tabi,^ VIa 
Anai^ysis of Variance; Tabi^e; for Mode^i, (6) 

^5* MS AMS 



mean 



nymm 



ny mm 



categories c—\ r^i^yi^—ymmY -^^tiiyi^—y^V 
within n — c (n—c)s^ s^ 



(i-^).' + (l-04 + ny-i 



+ r(r^ 



mean 

categories 

within 

(*) =-(i¥6'mean) 
n 



AC MS 
yA 



CMS 
(*) 



n \ m/ 



(MS categories) 



\N nMj 



(M^* within). 
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categories have the same slope (lay along the same line). 
For this would mean that the component of mean square 
for categories was zero (it is zero on the average only when 
the 7]i are the same), and this is a precise form of the rough 
statement that the categories are just alike. So we take the 
usual shaded triangle away from the triangle for categories. 

When N = M = oo , and the traces for categories and the 
mean fall in the same line, then the component of mean 
square for the mean is zero, and we conclude that the grand 
mean might be zero. Thus, we take both shaded and dotted 
triangles away from the triangle for the mean. When M is 
large, and many categories to be considered are not repre- 
sented in the experiment, we compare the mean square for 
the mean with the mean square for categories. 

Another extreme is N= cc, M = c, when the traces for 
the mean and for "within" must lie on the same line for the 
component of mean square for the mean to vanish. Here 
only the shaded triangle is taken away from the triangle for 
the mean. When all categories to be considered were repre- 
sented in the experiment, we compare the mean square for 
the mean with the mean square "within." 

And some situations fall in between, as the diagram illus- 
trates. 

Test Ratios and Confidence Limits 

We can again look for appropriate test ratios and confi- 
dence limits, with, the i^esults shown in Table VIb. 



Table VIb 
Test Ratios and Confidence Limits F'or Modeiv (6) 

Are there differences between categories ! 

r 



F = 



^MVi* yy ^ M>y categories 



s- MS within 

Might the mean equal Y? (M large) 

_ n(ym» — Y)^ _ MS mean 



F 



Myi»—yy 



MS categories 



t' 



Might the mean equal F? (M = c) 

n(ym9 — F)^ MS mean 



F 



s^- MS within 

Confidence limits for 3;aa? (M' large) 



3'»» ± tai 



{v; 



Myi^—yy 



n(c — 1) 
Confidence limits for ^aa? (M = c) 

(For c < ikf < CO combine MS within and MS cate- 
gories as suggested by AMS's of Table VI a.) 



The great difference in testing the mean — the great de- 
pendence on whether 

sampled categories = considered categories 
or 

sampled categories « considered categories 
shows up in more complex designs with avidity, subtlety 
and frequency. There it affects comparisons and is worthy 
of the analyst's best attention. 

DoUBIvE Cl^ASSlFlCATlON 

Basis 

Fifty years ago it was claimed that the way to run an 
experiment was to vary one thing at a time. If the nature 
of the subject is such that the results are not going to make 
sense, this is still the way to run an experiment. But, if in 
your subject the results make some kind of sense, it is 
usually much better to vary two things at once, or three 
things at once, or more. One of my friends is faced with an 
engineering problem where he is planning to vary 22 things 
at once. I don't advise you to start with that many, but he 
will learn more per dollar than if he varied one at a time. 

If it makes sense to set up a model like this, 

Jij = A -f 7;i + 4*3 + ^i} 

where i refers to the level or nature of one thing and ; to 
the level or nature of the other, where the plus signs are 
really plus signs, and the eij are really random fluctuations, 
then it is much more efiticient and useful to vary both things 
in a single experiment. 

If there is no semblance of a plus sign — if, for example, 
y increases when i increases for one value of ;*, but decreases 
when i increases for another value of / — then there may be 
little profit in varying two at once. There is not likely to be 
loss in varying only two at once, but more complex experi- 
ments (such as Latin squares, which we will not discuss) 
may burn the hand that planned them. 

But, fortunately, life is reasonably simple in most sub- 
jects. The plus sign will be a good approximation often 
enough for the use of such experiments to pay. It may not 
be gratis ; you may have to work for it. For example, the 
y's that you finally analyze may not be those with which 
you started. If you happened to be working on blood pres- 
sure, you may have to use the logarithm of the measured 
blood pressure; it is unlikely to be satisfactory to use the 
raw data in millimeters. For reasons that make sense when 
you think about them, factors that aflfect blood pressure 
tend to multiply together rather than add together in their 
effects, and then the logarithms are additive. 

Again the statistician ought to think hard about such 
matters. He ought to see the need for transformations. But 
sometimes the computing people may see something going 
on that will make clear to them that there ought to be a 
transformation. If the plot of the effect of one variable for 
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different values of another looks like Figure 7, for example, 
if the effect seems faster at higher levels, then we are a 
long way from a plus sign. The cure for this particular sort 
of deviation is to squeeze things closer together at the 
higher values than at the lower ones. You can do this by 
changing to the square root of the observed values, or to 
their logarithms ; one or the other may work. 

Things of this sort need to be kept in mind. The honesty 
of the plus sign controls the extent to which the observa- 
tions are adequately squeezed dry by this procedure. 

Model, Identity, Table and Diagram 

We shall treat this case briefly. A reasonable model for 
many uses is 

A fixed, 

]r]t\-a. sample from a population of size AT,, 

average t/a and variance or|, (6) 

^ <^y}- a sample from a population of size N^ , 

average <f>A, and variance (t| , 
-{ €ij \ a sample from a population of size N, 
average caa, and variance or. 
This is a case where the number of observations with i = io, 
y = yo is g{h)h{j^) — in fact g{i) = 1 = h(j)— so that the 
^'-classification and the /-classification are orthogonal. This 
simplifies matters considerably. 




Figure; 7 



One identity is 

tyfi = ty* + tiyi»—yy + tiy—yy 

-{- t(yij—yi»—yj+yy 

+ (r-l){c-l)s' (&) 

+ ^tylj -Itjy'^^ -Itiyt-^y^y^. } • 

The analysis of variance table is given as Table VI I a, and 
the diagram as Figure 8. The two classifications are con- 
veniently referred to as "columns" and "rows." 
The details of this model and those of many more compli- 
cated ones we must leave to the reader's thought and study. 
No two books will give him the same account, but a few of 
interest are given in references 14-18. 



Item 



Tabids VII a 
Anai^ysis oe* Variance; Tabi^e; i^or Mode;i, (7) 

DF SS MS 



CMS 



mean 


1 


rc{ymm—Y)^ rc(ym»—Y)^ 


n 


columns 


c-1 


rMyf—y»»y ^_^ti(yi*—y**y 


-(MS columns) — - s^ 


rows 
residue (c- 


r-l 

-l)(r-l) 


ctiiyj—yy ^_i tj(yj—yy 

(c—l){r-l)s^ j2 


-(MS rows) s^ 

c^ c 






AMS 


ACMS 


mean ( 1 — 

columns 

rows 

residue 


f)»i + (i 




-YY {y..-Yy 

2 



(*) = -^ (MS mean) ^ (c""]^) ^^^ columns — MS residue) 

— —>( — Tp) {MS rows — MS residue) — I ^ j {MS rows — MS residue) 

*Is most easily found numerically from AMS mean, CMS columns, CMS rows, and CMS residue. 
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(C-l)(r-l) I C-1 I r-1 I 1 
Degrees of Freedom 



Figure 8. Anai^ysis of Variance; Diagram 
FOR MoDiii, (7) 
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DISCUSSION 

Mr. Keller: In our steam turbine testing work we have 
found, by running duplicate tests, or tests on two units that 
are duplicates, that we obtain from one-half to one per cent 
unexplained variation. It is quite important, in our design 
work, that we take advantage of differences that may 
change the performance to the extent of one or two-tenths 
of a per cent. 

In our talks with statisticians they usually point out that, 
if we use Latin squares and change several variables at once, 
we could obtain the required design information at smaller 
cost and with fewer tests. I should like to ask whether the 
question of how you should plan your experiment is affected 
by the expected difference that a change in design would 
cause, relative to the unexplained difference between two 
tests on the same unit. In other words, the standard devia- 
tion is one per cent, and you want to test for three or four 
items, each of which might amount to two-tenths of a per 
cent ; do the standard methods still apply ? 

Professor Tukey: Yes. This is entirely typical of agricul- 
ture, where these methods were first developed, because 
that is where they first realized they were in serious trouble. 
An alteration of one per cent in the yield of barley in Ire- 
land means a considerable number of pounds, shillings, and 
pence to the Irish. It is awfully hard to get any one field 
experiment to have a fluctuation as low as 10 per cent, and 
these methods were developed just to get at that sort of 
situation. If you are interested in only ten per cent, and 
your experimental error is one per cent, it doesn't matter 
how you do it ; you will find out. But if you have to work 
for it, things of this sort are indicated. Whether you want 
to use Latin squares or not is another matter. You have to 
know a lot about the situation; and I don't know about 
steam turbines. So I can't tell you whether you are to use a 
Latin square or not. But I think that you would find some 
design, more complicated than the one you are probably 
using, is likely to help. 

Mr. Keast: In the example that you have shown, where 
your function jij is affected by i and /, and the diagram 
underneath, which shows greater variation at the right- 
hand side than at the left-hand side — what is the essential 
point of the diagram ? 
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Professor Tukey: The essential point which I tried to 
make in this diagram, is that there is greater variation at 
higher levels, rather than that it happens to be at the side. 
As the level of the original 3; increased, the differences be- 
came larger. If that is the case, you have some hope of con- 
trolling it by going to the square root of 3; or the logarithm 
oiy. 

Mr. Keast: My problem is, if you don't know the varia- 
tion in the first place, is it not that you are considering the 
variation with each factor to be linear over the range with 
which you are working ? That is, in planning an experiment 
where you had everything vary at once, are you assuming a 
linearity there ? 

Professor Tukey: No, very definitely not, because the re- 
mark I made would apply perfectly well if the situation 
went as follows : in the regions where y is low, the differ- 
ences are small; and in the region where y is large the 
differences are large. You are talking about a plus sign in 
the way that different things interact ; but you let rji be any 
function of i it chooses, and you have allowed <f>j to be any 
function of jf it chooses. The problem is to make the inter- 
action behave, and you can let the individual variables run 
as they choose to make things go. 

Dr. Lotkin: I would like to ask two questions pertaining 
to some of the work we are doing at the moment, dealing 
with measurements of angles such as you come across when 
you contend with theodolite data. 

In smoothing such data we have a choice of selecting suc- 
cessive groups of data. The question arises: how large 
should you take such groups in order to obtain feasible fits ? 



Because we have found that, depending on the size of the 
groups you take, you get slight variations in the fit. 

Second, in doing this smoothing by means of orthogonal 
polynomials, the degree of the polynomial will vary on your 
significant answer. In planning this for the machine, we 
have a choice, then, of either varying your degree of the 
polynomial — which can become quite involved — or adher- 
ing to a certain fixed prescribed degree. 

Now, we are aware of the fact that, if we take a fixed 
degree for this polynomial, we might run into some danger 
of over-smoothing the data. What I would like to know is 
if this danger is not, possibly, over-emphasized. 

Professor Tukey: I don't want to try to answer this ques- 
tion in detail, due to time limitations, but I can say some 
things about it. Basically, you have a problem where you 
are getting data out of a process. You have some theodo- 
lites, and you hope they run about the same from day to 
day ; and what you are going to do with this ought to de- 
pend on a whole backlog of experience, and not what you 
obtained on this particular run, generally speaking, unless 
you have evidence that this run is out of line in some way. 

What is needed here, then, is to find out the essential 
characteristics of the situation, and make up your mind 
what smoothing you want to do on the basis of that — not 
just to apply some sort of test to this strip of data and work 
with it accordingly. 

You are raising the question, really, of how should this 
kind of data be analyzed. How should one analyze the data 
on soybeans compared to data on potatoes? That requires 
going back and looking at the essentials of the data. 

I think that trying to get at the power spectrum is the 
way to find out what you want to do in this case. 
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THE generally- recognized machine methods which have 
been adapted to statistical calculations were first outlined 
by A. E. Brandt^ in 1935, although some of the methods 
are known to have been in use before then. Dr. Brandt de- 
scribed the use of progressive digiting methods to obtain 
sums of squares and sums of products which were re- 
quired in the statistical analyses. Since that time little has 
been added to the methods, save some improvemeints in 
details as a result of the steadily increasing efficiency of 
the newer models of machines. The following is essentially 
a description of the applications of the progressive digiting 
methods. 

The methods of analysis of variance and multiple regres- 
sion are a part of what have been called "standard methods 
of analyzing data." The two methods are closely related 
mathematically, i.e., the analysis of variance can be re- 
garded as a special case of multiple regression in which the 
independent variables are arbitrary or dummy variates. It 
is usual, however, to think of the analysis of variance as 
concerning itself with a single variable, while the purpose 
of multiple regression is to relate one or more dependent 
variables with two or more independent (or "causative") 
variables. In either case it is conceptually easy to regard 
either problem simply as a breakdown of the "total varia- 
tion" in a single variable into several component parts, re- 
gardless of how this breakdown is accomplished. 

The example chosen for this paper came from a problem 
where both of the above-mentioned techniques were found 
useful. 

In a continuing project at the North Carolina Agricul- 
tural Experiment Station the attempts to improve on the 
present varieties of peanuts involve experiments embracing 
large numbers of measurements on individual plants. Con- 
sider, for example, an experiment made up of four different 
crosses from which were selected seven different strains. 
From each of the 28 seed stocks plantings were made to 
allow the measurement of ten plants, and each seed stock 
was planted in five different replications (locations). This 
kind of an experimental design is called, in statistical par- 
lance, a "randomized block." 



The model for this design may be written as 

yijk = i^ -\- pi-\- yj-\- $jk + (py)ij + {pi)i}k + ^mi- 
fi = unspecified mean parameter 
Pi = effect of ith. replication (i — 1, . . . , 5) 
yj = effect of yth cross (/ = 1, . . . , 4) 
ijjc = effect of ^th strain in the /th cross 
k = ( 1, . , . , 7) for each / 
(py)ij = effect of interaction of jth cross with ith repli- 
cation 
(pi)ijk = effect of interaction of ^th strain with ith repli- 
cation for each of / crosses 
eijki = a random error NID(0, o-^) associated with 
each plant (/ = 1, . . . , 10). 

The analysis of variance, with associated degrees of free- 
dom, is derived from the model, each line in the analysis 
being associated with the indicated parameters of the model. 

AnaIvYSis of Variance; 



Source of Variation 




Degrees of Freedom 


general mean 




1 


C.F. 


replications 




4 


SS(R) 


crosses 




3 


SS(C) 


replications X crosses 




12 


SS(RC) 


strains within crosses 




24 


SS(SC) 


replications X strain within crosses 


96 


SS(RSC) 


individual plants within 


plots 


1260 


SS(IP) 



Total 



1400 



SS(T) 



The sums of squares for each of the above effects may 
then be segregated. 



= C.F. 



1. General mean: (l/ijkl) ( / j v ) 

2. Replications :(-|^^X](SyY-C-^- = SS(R) 
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3. Crosses:(^)X^(2]yY-C-F- = SS(C) 

4. R X C (^)l]('X)yY - C.F. - SS(R) 

-SS(C) = SS(RC) 

5. Strains in crosses : 

6. Replication by strains in crosses : 

7^(2]^ J-|7Z)(5yy~^^^^^^ ^ ss(Rsc) 

7. Individual plants within plots : 

E/-7E(EyY=SS(iP) 

8. Total: 2^/ = SS(T) 

[Note: The abbreviated notation used here avoids the bulky multi- 
ple summations, and the term C.F. refers to the "correction factor" 
for the general mean. The term SS(R) is read "sum of squares 
for replications," etc.] 

The card used for the analysis is indicated in Figure 1. 
Each classification is punched in a separate field, as is each 
variable to be considered in either the analysis of variance 
or the regression analysis. 

The analysis is produced by successive sorting, summing, 
and progressive digiting operations, interspersed with oc- 
casional use of a desk calculating machine. Detailed steps 
follow : 

1. Sort on "strains," "crosses," and "replications." 

2. Tabulate and summary punch with controls on the 
three sorts. 

3. Compute / j I / ^ y | by progressive digiting with 

ijk \ I / 

the summary deck. 

4. Remove the controls on "strains" from original deck 
and tabulate again. 

5. From the tabulation in (4) compute / ^ [ / ^ y j 
with desk calculator. *^' \ ''^ ' 

6. From the same tabulation in (4) the totals 

Z(Zy)andX)(Zy) 

} \ ikl / i \ jkl / 

are obtained with which to compute SS(C) and 
SS(R), respectively. 

7. The correction factor C.F. is computed from a final 
total of the tabulation (4). 

8. The total sum of squares, SS(T), is then computed 
with the original deck, using the progressive digiting 
technique as before. 

9. Each line of the analysis of variance is then derived 
from the above quantities by applying the proper 
divisors and making the subtractions indicated in the 
formulas already given. 



0000000000000000000000000 

1 2 3 4 5 6 7 8 9 10 1112 13 14 15 ie 17 18 19 20 21 22 23 24 25 
1111111111111111111111111 

2222222222222222222222222 
3333333333333333333333333 
4444444444444444444444444 
5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5| 
666666 88 68 666866 6 6 6666666 
7777777777777777777777777 
8888888888888888888888888 
9999999999999999999999999 

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 IE 17 18 19 20 21 22 23 24 25 



Figure 1 



While the above procedure may not be the shortest pos- 
sible method for any one instance, it seems clear that such 
a process is admirably adapted to routine computing and 
requires a minimum of supervision. 

The; Mui.tipi,e; Re;gre;ssion Anai^ysis 

In the same study it was necessary to compare the in- 
herent (genetic) relationships between several of the plant 
characteristics with the observed (phenotypic) correlations. 
For these purposes are required the sums of squares and 
sums of products of all variables under consideration. 

The phenotypic correlations are defined, simply, as the 
product-moment correlations between the variables con- 
cerned. That is, if x^ and x^ are the variables, then the 
phenotypic correlation r^ is given as : 

_ l^{x,-x,){x^—x^) 

The same quantities appear in a multiple regression 
analysis of the regression of a dependent variable, yi, on 
several, p, independent variables Xj. Suppose the regression 
of yield on the variables x-^ = seed/pod, X2 = pods/plant, 
and x^ = wt./seed were desired. The model, /> = 3, 

(yi—y) = b,(x,i-x^) + &2(-^2i-^2) 
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represents the observed data where ei is an estimate of the 
measurement errors involved. In ordinary least squares the- 
ory, the solution for the coefficients b,- proceeds as follows : 
In matrix notation if 



and 



then 



%fc =2^(^ji-^j)(^ki—^k), ij,k = 1,2,3) 



b} = [Ofcl [dk] where [cjk] = [%fc]"^ • 
[Cjjc] is also called the covariance matrix because it contains 
the coefficients for computing the variances and covariances 
of the b/s, e.g., V{bj) = CjjS^. Then the reduction due to 

regression is given by y bjQj, and the residual sum of 
squares to estimate the measurement error is 

2^^yi~yy ^ 2li^'^^ = {n - p - i)s\ 

where {n — p — \) represent the degrees of freedom in s"^. 
It is clear, then, that both types of analyses require the 
computation of a number of quantities of the form 

/ .\-^Ji ^i)\^ki ^k) ) 

and these are easily obtained from a progressive digiting 
scheme using only a sorter and accounting machine, al- 
though use of a summary punch will shorten the time of 
calculating somewhat. 

In usual practice it will be desirable to set up the entire 
group of variables in the counters and successively mul- 
tiply the entire group by each variable in the group. This 
procedure gives a check on the machine work, since the 

y XjXjc =y x^Xj, the two operations being accomplished 

independently. 

The matrix inversion required in the regression analysis 
is a more difficult thing to accomplish without special equip- 
ment. It is sufficient here to mention that this can be done 
with the IBM Type 602 Calculating Punch, but the process 
is quite involved. This job is, perhaps, one best done on 
the newer models which have been demonstrated at this 
seminar. 

The foregoing discussion was intended as a brief sum- 
mary of the application of IBM equipment to statistical 
analysis. Only a few of the basic operations were described. 
To anyone familiar with both the analytical and computa- 
tional problems many short cuts and improvements will 
suggest themselves — especially in particular problems.^ 
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DISCUSSION 

Mr. Dye: There are several other ways, in which you 
might be interested, by which you can derive the same re- 
sults that have just been described, by use of standard IBM 
equipment. I think they might be of passing interest, if 
nothing else. You can, with small volumes, on a 604, ac- 
tually square the individual components and obtain the 
cross-products of the various variables. Then, by running 
the cards through an accounting machine, add them and 
obtain the summations in printed and punched card form. 
By running the cards through the 604 again, actually the 
means are derived, the standard deviation squared, and in 
some cases the actual r squared. 

As far as progressive digiting is concerned, there are, 
also, two methods which are available, depending upon 
what kind of equipment is present. If the machine contains 
the card cycle total transfer, you can do up to 40 digits by 
sorting on the complete item, transferring at the end of 
each control break enough items so that, when you are fin- 
ished, you will have on your minor breaks, a summation of 
the terms themselves, and on the major, a summation of 
their squares. 

Chairman Hurd: I might make a note here of something 
that Mr. Bailey told me. He indicated that he had prepared 
a control panel for the 604 which would handle all analysis 
of variance procedures up to a reasonable three classifica- 
tions, with one pass of the machine. 

Mr. Bailey: The technique that we generally use is to 
prepare one card for each item in the table. We prepare one 
card for each of those ten items through all the replications. 
Ordinarily, we subtract a base from the items and code 
them to reduce the number to a reasonable size. We have it 
set up so that, on one card, we can handle numbers with as 
many as four or five digits, subtracting a base from each of 
those numbers. In one pass through the 604, the base is sub- 
tracted from each of the five-digit numbers, then the differ- 
ences and the squares are summed and punched in the detail 
card. We sort the detail cards on our first classification, 
summary punch, repeat the process for the second classifi- 
cation and the third classification, then sort on two classi- 
fications at a time, summary punch, and finally summary 
punch with all controls off. Next, the summary cards and 
the detail cards are put together and fed into the 604, using 
the same control panel, with a switch where each of the 
sums is divided by the number of items represented on the 
summary card and obtain the means. Multiply the mean by 
the sum again to obtain the correction factors. 

All these correction factors and sums of squares are listed 
both for summary cards and detail cards, and it is a simple 
matter, just by a process of subtraction, to obtain all the 
sums of squares in the analysis of variance table. 
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Mr. Clark: There is a method of obtaining sums of 
squares and cross products, called digiting without sorting, 
in which the multiples are represented as combinations of 
1, 3, 5, the same way that you would a binary code 1, 2, 4, 8. 
In other words, 9 is 5 + 3 + 1, 8 is 5-f- 3, and so on. If 
there is enough capacity you can, at the same time, digit for 
10, 30, 50, and then 100, 300, 500. Really, there are four 
methods : the multiplication, the summary punching, the 
card total transfers, and the digiting without sorting. And 
one should always weigh the economics of which way to do 



it. However, you can make a general statement to the effect 
that, if you have a large number of variables and a large 
amount of data per dollar of computing, your best method 
is summary punching with the method that you outlined. 

Mr. Belzer: The first method you described requires 
three counters for each bank ? 

Mr. Clark: That is right. It is an extravagant method. 
The beauty of it is that when you have an enormous number 
of cards, a small number of variables, you don't want to 
sort. 
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MATHEMATICIANS are working with numerical 
constants. Once a specific problem is solved, the same for- 
mula will apply to similar problems. This is not true when 
multiple variables are introduced from problem to problem. 
We, in the healing arts, deal with such a great number of 
variables within the human body, as well as from individual 
to individual, that it has not been possible to apply mathe- 
matics to the human race as a mass ; trends, indications or 
approximations are the best we can hope to obtain. 

Our IBM installation consists of a Type 16 Motor Drive 
Duplicating Punch. The problems we have encountered 
have been merely to standardize methods and procedures of 
collecting data, proper coding and punching, so that the 
IBM Service Bureau can compute averages or percentages 
on several pertinent items. 

Our field research program is now in its third year and 
is a cumulative study. With each passing year, the study 
will become more useful as the number of cases in each 
diagnosis increases. The field research data booklet contains 
the information concerning the cases compiled. The first 
year, 1947-1948, this program was in operation we studied 
700 cases with 16 diagnosed conditions. By longhand meth- 
ods it took us 800 man hours to calculate our data. During 
the year 1948-1949, we processed 3,400 cases on 38 diag- 
nosed conditions in 400 man hours by IBM methods. This 
is exclusive of the two hours required by the Service Bu- 
reau to tabulate the data. What this amounts to, roughly, is 
4^ times the work load in one-half the time required by 
longhand methods. 

Since this program was started previous to switch-over 
to IBM cards, and since it was a cumulative study, it be- 
came necessary to have our codes made up into rubber- 
stamp form in order to bring our previous cases up to date. 

The case history of each patient studied in the research 
program is recorded on a form such as Figure 1, page 118. 
This wealth of information may be placed on one IBM card, 
as indicated in the right-hand side of the figure. Each case 
is coded as to: 

1. Industry. There are 13 classifications which indicate 
the field of work in which the patient is engaged. 
Thus, percentages in each different type of industry 
may be determined. (At a later date if we should re- 



quire data on specific types of industry, special studies 
may be conducted.) 

2. Occupation. The type of activity the patient pursues 
is indicated by 11 categories. The occupation code 
enables the determination of percentagewise distribu- 
tion. 

3. Injury. Ten classifications indicate the nature of the 
injury, while 16 other classifications give the way in 
which the injury was incurred. 

4. Chiropractic Analysis. The analysis of the patient's 
condition after spinal analysis is coded into one of 16 
categories. 

5. Diagnosis. The coding of diagnosis of the patient's 
condition consists of merely assigning 1 to anemia, 
2 to angina pectoris, 3 to arthritis, 4 to asthma, etc. 

6. Patient's Condition. This is coded for before chiro- 
practic care, with chiropractic care, and after chiro- 
practic care. 

7. Insurance. Information as to whether claims were 
paid, the compensation involved, and the type of in- 
surance policy. 

Other necessary information for research analysis such 
as case number, age, sex, days under medical care, number 
of office visits, number of X-rays, and number of days 
(working) lost before chiropractic care and while under it 
is in actual numbers. 

The Service Bureau sorts like numbers together accord- 
ing to diagnosis, and from there on it is a simple matter of 
tabulating like data in the same columns, with totals, so 
that we can obtain various data and averages or percentages 
such as : 

1. Average age. 

2. % females — % males. 

3. Average number of days under chiropractic care. 

4. Patients' condition at end of chiropractic care. 

5. %well. 

% much improved. 
% slightly improved. 
% same. 
% worse. 

6. Average number of years the diagnosed condition had 
existed previous to chiropractic service. 
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DO NOT WRITE IN 
THIS SPACE 



INTEHNATIONAL CHIROPRACTORS ASSOCIATION 

INDUSTRIAL RESEARCH (Form IR4) Revised 



1> Name or Case No 

2. Employer 

3. Age Sex 

4. Nature of Injury 

5. How Was Injury Incurred 



Date / / 19. 



Occupation 



6. Type of Work Being Performed When Injured 

7. Date of Injury / /19 Time of Day 

8. Date Reporting to Chiropractor / /19 Time of Day .............. 

9. Date of Discharge / /19 

10. Number of Days Under Chiropractic Care Number of Office Visits 

11. Patient's Condition After Chiropractic Service _ 

(Well, improved, temporary partial disability, temporary total disability, permanent 
partial disability, permanent total disability, or other.) 

12. Chiropractic Analysis __ 

13. Number of X-rays for Chiropractic Analysis 

(a) Before Chiropractic Care 

(b) Under Chiropractic Care 



14. Number of Days Lost From Work 
on This Case: 



15. Number of Days Under Medical 
Care, If Any, In Connection With 
This Injury: 

16. X-ray Costs 

17. Service or Adjustment Costs . . 



(a) Before Chiropractic 

(b) With Chiropractic .. 

(c) After Chiropractic . 



Total Costs $. 



18. Name of Insurance Company 

19. Was Qaim Paid In Full - 

20. If Additional Information Given on Back of This Sheet, Check This Space 

□ Industrial Case 
Claim Paid by Insurance Company 



D 

Claim Denied by Insurance Company []] 

□ Industrial Case 
Claim Not Presented for Legal Reasons 

I — I Health & Accident Policy — Claim Paid Q 



WHEN COMPLETED MAIL TO: 

INDUSTRIAL RESEARCH, 
ICA, 838 Brady Street, 
Davenport, Iowa 



u 



(Not Industrial) 



C9aim Denied (~| 



Chiropractor — 

Address City State ..._.. 

PLEASE CHECK ALL INFORMATION FOR ACCURACY 
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Transforming Theodolite Data^ 

HENRY SCHUTZBERGER 

Sandia Corporation 



A THEODOLITE is a precise instrument for measur- 
ing the azimuth and elevation angles of an object. It is de- 
sired to reduce these angular measurements, arising from 
two or more theodolites, to the rectangular Cartesian co- 
ordinates of the object. The quantity of computation for 
several hundred observations from more than two theodo- 
lites becomes overwhelming when these calculations are 
performed on hand calculating machines. However, the 
computation for these same observations becomes quite 
feasible with automatic computing machines. The method 
discussed here has been used with as many as five theodo- 
lites with much more satisfactory results than previously 
used two-theodolite procedures. 

The instruments most generally used to obtain the ob- 
servations are the Askania theodolites. They have proved 
to be the most valuable of the cine-theodolites available. 
Captured from the Germans after the war, they are used 
extensively on test ranges in this country. Attempts have 
been made to duplicate them, but with little success up to 
the present time. 

Theodolite Instrumental Brrors 

The Askania cine-theodolite, like any other high-preci- 
sion instrument, is subject to many errors. Frequently these 
errors arise, not from any inherent defects in the instru- 
ment, but from the fact that the instrument can be read 
more accurately than the adjustments of the instruments 
can practically be made. 

The errors to which the Askania is subject are : 

1. Tracking Brrors. These are not really errors in the ordinary 
sense but arise from the fact that the operators are not able 
to keep a moving object exactly on the center of each frame 
of the film. Thus, it is necessary to correct for this displace- 
ment on each frame. 

2. Orientation Error. This error occurs because the instrument 
is not oriented to read the proper predetermined elevation 
and azimuth angles when set on a given target. 

3. Leveling Error. This error occurs when the base plate (azi- 
muth circle) of the instrument is not exactly level. The base 
plate error consists of two parts : the angle of inclination of 
the base plate with the true horizontal plane, and the azimuth 
direction of the intersection of the base plate and this hori- 
zontal plane. 

*This paper was presented by Kenneth C. Rich. 



4. Collimation Error. This error occurs when the line of sight 
down the instrument telescope is not exactly perpendicular to 
the horizontal axis of the instrument. 

5. Standard Error. This error occurs when the horizontal axis 
of the instrument does not lie exactly parallel to the base 
plate of the instrument. 

6. Tilt Correction. Because the local zenith at the instrument is 
not parallel to the zenith at a removed origin, owing to the 
curvature of the earth, this correction must be applied. 

7. Refraction Error. This error is due to the bending of light 
rays when passing through media of changing density. 

8. Scale Error. The Askania cine-theodolite has an extremely 
precise scale, but the optical system of several prisms, trans- 
mitting the scale to the film, may be out of adjustment and so 
introduce an error in the scale reading. 

9. Bearing Error. As the instrument is rotated through the azi- 
muth, its weight is supported by a main thrust bearing. Any 
irregularities in this bearing, or in the ways in which it rises, 
introduces an error in the elevation angle. 

For the accuracy of measurements desired, each of these 
corrections must be taken into account. At present these 
corrections are made by hand computations, as it was 
not considered efficient to perform them on mechanical 
equipment. However, a device built by the Telecomputing 
Corporation, known as an Askania Reader, has been or- 
dered. This machine, which is connected to an IBM Type 
517 Summary Punch, enables an operator to make the nec- 
essary measurements on the film, and records these meas- 
urements and the instrumental constants automatically on 
an IBM card. With these data on cards, it will be possible 
on the IBM Card- Programmed Electronic Calculator to 
make all necessary corrections. 



A Solution Used in the Past to the Two-theodolite Problem 

Let S = azimuth angle measured from the positive X 

direction 
Let H = elevation angle 
— XYZ = right-handed reference frame in which Z is 
vertical 
X,Y,Z= space coordinates of object 
x,y, s = space coordinates of observation point 
Subscripts 1 and 2 = quantities pertaining to theodolite 
1 and theodolite 2, respectively. 
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COMPUTATION 



The usual relations yielding X, Y, and Z from a pair of 
theodolite observations are set down for reference : 



X = X, + 



jx^-x^) tan^a - (y2-yi) 



tan 5*2 — tan 6*1 
F = Vi + (X-x^) tan^i 
tan H^ 



(1) 
(2) 



Z = ^,-\-{Y-y,y 



sin 5*1 



= Z,+ (X-rO^-^(3) 



Z = ^, + (F-3,,)|^ = Z, + (X-x,) 



cos 6^1 

tan //a 
cos ^S", 



.(4) 



The order in which the preceding relations are given is 
convenient for computations. Under certain conditions, it 
is necessary to change the form of these relations,^ but the 
significance of the method remains unchanged. Relations 
(3) and (4) give the same value of Z only when the lines 
of sight make a true intersection. 

It will be noted that a redundancy exists, in that four 
quantities (S^, H^, S^, H^) are given from which three 
quantities (X, F, Z) are to be determined. Except when 
the lines of sight make a true intersection, this problem has 
no proper solution. 



Derivation of a Least-squares Method of Data Reduction 

The system and angles are defined in exactly the same 
manner as before. The direction cosines of the line of sight 
of each theodolite may be determined from the H and S 
angles and are : 

h = cos Hi cos Si 
Mi = cos Hi sin Si 
ni = sin Hi 
where the sub- 
script i denotes the 
number of each 
theodolite. For 
convenience in no- 
tation, the space 
coordinates of the 
ith theodolite shall 
be denoted as (xi, 

Figure 1. Diagram Ii^lustrating Geometry 
FOR Derivation oe Direction Cosines 



i^ShouId 5*1 approach 90 degrees, i.e., X — x± be very small, it is 
better to compute Y — yx from a relation similar to Equation 1 but 
involving cotangents of the angles, and then to compute X — xi 
from a relation similar to Equation 2. 




p„(x„,y„,z„) 




The coordinates of 
any point lying on 
the ith line of sight 
may be expressed 

Piix^ja,) as: 

■A ;( = Xi — p iiSi 

Yi = yi + '^iSi 

Z^i '^ Zi ~p niSi 

where Si is the dis- 
tance along the line 
of sight from this 
point to the the- 
odolite at {xi, yi, 

Zi)' 

Figure 2. Diagram Showing Lines oe Sight 
AND Location oe Desired Point 

Now, if the several theodolites are pointing at a fast mov- 
ing object in space at a considerable distance, the lines of 
sight in general will be skew with respect to each other. Let 
the coordinates, to be determined, of the object be denoted 
by Po(-^o> ^oj ^o)- From the point Po> consider the con- 
struction of lines perpendicular to each line of sight. Denote 
the intersection of each line of sight with its perpendicular by 
PiiXi, Yi, Zi). By this notation, then, the distance from 
the point Pq to each line of sight may be expressed by 
rf? = {X(,-Xi-liSiy + (Yo-yi—miSi)^ 

-}- (Z^-Si—niSiY (5) 

For the determination of the point P^ which best fits the 
lines of sight, a least-squares approach is believed to give 
the closest representation of the actual condition; i.e., the 
sum of the squares of the distances from this point to each 
of the lines of sights is to be minimized. Thus, since in 
equation (5) the values of (xi, ji, Si), the ith theodolite 
coordinates, and k, nii, ni, the direction cosines of the line of 
sight from the ith theodolite, are determined by the position 
of the theodolite and the direction in which it is pointing ; 
the values of {Xq, Fq, Zq) and Si are the variables which 
may be changed to minimize the sum of the squares of the 
distances from point P^ to the lines of sight. Thus, 
dl=f{X„Yo,Z„Si). 

A set of direction numbers for the ith line of sight is 
li, Mi, ni, and a set of direction numbers of the line joining 
the point in space (Zq, Yo, Zq) to any point {Xi, Yi, Zi) 
on the ith line of sight is X^ — Xi, Y^—Yi, Z^ — Zi, or 

-^0 ^i ''i^h -'0 yi '^iSi, Zq Si WjJi. 

A necessary and sufficient condition that these lines be 
perpendicular to each other is that the sum of the products 
of corresponding direction numbers be zero ; i.e., 

li{Xo-Xi — ksi) -{-mi(Yo — yi — MiSi) 

-f- ni{Zo-Si—niSi) = (6) 
Solving for Si 

Si = li(Xo-Xi) + mi{Yo-yi) + «i(Zo-^i) . (7) 
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Thus, the parameter Si may be eliminated by making use 
of the condition of perpendicularity. 

Substituting this value of Ji in (5), letting liXi + miyi + 
fiiZi = pi, and simplifying 

d'i = [(!-/?) X, - kmiY, - kniZ,~Xi + kp^y (8) 
+ [-/iWiXo + {l — m'l) Fo - WiWiZo — yi-\r niipiY 
+ [-liUiXo - niiniY^ + (l-«l) ^o --^i + %/'*]' • 

In order to minimize the sum of the squares of the dis- 
tances to each line of sight, i.e.. 



7 d'i = P (X^, Fo, Zo) = min. , 

t=i 

the following condition is necessary : 

dF_^dP_^dF_ 

dX'dY, dZ,-^ ■ 

Summing di for all theodolites 

n 

^d=P(X„Y„Z,) = 



(I-42) Xo - /iW^Fo — hfiiZf, - Xi + lip 



(9) 



(10) 



(11) 



+ - /jWiZo 

[ 



] 



+ (1-w?) Fo — WiWjZo — 3'i + Wi/), 



+ - liHiX^ - miUiYo + (l-«?) Zo - Si-{- nip 



J 



Taking a partial derivative with respect to X^ and sim- 
plifying 

n n n 

" i=l i=l i=l 

-^^i+^hpi = 0. (12) 

imilarly : 

n n n 

= ^(-^iWi) Xq+Z /I — wf) Fo + /^(-W{^0 Zo 

i=l i=l i=l 

-^yi+^^ipi = ^- (13) 



aF. 



az. 



W OT »■ 



1=1 t=i 

» n 



-^^i+^«i/'i = 0. (14) 

i=l i=l 

Rewriting (12), (13), and (14) 

n n n 

^(1-/?) Zo +^(-/,mO Fo +Y^{-kni) Zo 



1=1 i=i 

n n 



= X]-*-i-X]^^^^ (15) 

i=l t=i 

w n n 

2_](-4wO Zo -h^(l-w?) Fo +2J(~^^^*) ^" 

i=l i=l 

n n 

i=l i=l 

n n n 

^{-km) Xo -f^(-m,MO Fo -f-2^(l-w!) Zo 



i=l i=l 

1=1 i=l 



1=1 {=1 



t=l *=1 



= 7 / { —7 v^^/'-i 

i=l i = l 

Thus, an array of three symmetric simultaneous linear 
equations in the variables Xo, Fq, Zq is obtained. All other 
values are obtained from theodolite data. 



An Outline of the Abbreviated Doolittle Method of Solution 
of a System of Symmetric Linear Equations 

There are many methods for solving systems of linear 
equations such as those given in equation (15), and it was 
felt, after several methods were investigated — inasmuch as 
automatic calculating equipment would be available — that 
the abbreviated Doolittle method was the most economical 
in time for the desired accuracy. Dwyer's method, known 
as the Abbreviated Method of Multiplication and Subtrac- 
tion-Symmetric or, as he calls it, the Compact Method, is 
somewhat shorter than the abbreviated Doolittle method, 
but it involves more difficult 602-A control panel wiring.^ 

This method is applicable to any number of theodolites, 
greater than one, with no changes. Because, occasionally, 
tracking operators lose the object, this is a very necessary 
condition. 

Approximately 45,000 arithmetical steps, depending upon 
the number of observations, are involved in the data reduc- 
tion for one test. 
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Chart I 

OUTLINE OF ABBREVIATED DOOLITTLE METHOD OF SOLUTION 

/ = cos H cos S 
m = cos H sin S 
w = sin// 
p = Ix -\- my -\- ns 





A 


B 


C 




D 


1 


(lA) =-2(1-/2)* 


(IB) ==— S/w 


(IC) =-S/« 


(ID) 


= -Lx—np 


2 


same as (IB) 


(2B) =:S(1-ot2)* 


(2C) = — 2mw 


(2D): 


= liy — "Zmp 


3 


same as (IC) 


same as (2C) 


(3C) =2(l-n2)* 


(3D) 


= 2^ — lifip 


\ 


(4A) = (lA) 


(4B) = (IB) 


(4C) = (IC) 


(4D): 


= (1D) 


5 


(5A) - 1 


^^^^ (lA) 


(5C) ^1^> 


(SD): 


_(1D) 
(lA) 


6 




(6B) = (2B)-(5B)(1B)* 


(6C) = (2C)-(5C)(1B) 


(6D) = 


= (2D)-(5D)(1B) 


7 




(7B)=1 


j7 (6C) 
^^^^ (6B0 


(7D) = 


(6D) 
(6B) 


8 






(8C) = (3C)-(1C) 
(5C)-(6C) 
(7C)* 


(8D) = 


= (3D)-(1C)(5D) 
-(6C)(7D) 


9 






(9C) = 1 


(9D) = 


_(8D) 
(8G) 



10 Z=(9D) 
F=(7D) 
Z=(5D) 



(7C)Z 
(5C)Z- (SB)F 



*Al,WAYS POSITIVE. 



The time required to compute by hand a reduction, of all 
the data obtained from a complete test, by the abbreviated 
Doolittle method is prohibitive. It is roughly estimated that 
two experienced persons might be able to compute the co- 
ordinates for one complete test in about a month. By use of 
the IBM equipment now at hand, which includes two type 
602-A calculating punches, this process, starting with the 
film, can be completed in approximately three days. How- 
ever, by use of the Telecomputing Askania Reader and the 
IBM card-programmed electronic calculator, now on order, 
it is estimated that a reduction of one complete test from 
data of five theodolites may be completed in a matter of 
hours. 

In regard to a two-theodolite solution versus a five- 
theodolite solution, a study has been made to answer the 
questions as to which method produces the better solution, 
and how much better is this solution. In using a comparison 
of third differences as a measure of the random errors 



present, it was found that a five-theodolite solution was 
considerably smoother : in fact, it had only about 50 per cent 
as much random error as did the two-theodolite solution. 



1. Paul S. DwyEr, "The Solution of Simultaneous Equations," 
Psychometrika, Vol. 6, No. 2 (April, 1941). 



DISCUSSION 

Mr. Rich: Mr. Schutzberger's problems are very similar 
to those of the Naval Ordnance Test Station, and I'm sure 
that similar methods are used by many groups in the coun- 
try. The concern of groups involved in data reduction is the 
speed with which results may be obtained after the tests 
and then placed in the hands of the interested parties. 

Professor Tukey: A number of years ago we were trying 
to do things like this with two theodolites, using Mitchell 
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Chart II 

OUTLINE OF COMPACT METHOD OF SOLUTION 
(Abbreviated Method of Multiplication and Subtraction — Symmetric) 

/ = cos H cos S 
m = cos H sin S 
n = sin H 

p — Ix -\- my -\- nz 





A 




B 








C 


D 


1 


lA 
2(1— ;2) 




IB 








IC 

— 2/n 


ID 

2;r-2//. 


2 






2B 

S(l-W2) 








2C 


2D 

Sj; — 2wi^ 


3 














3C 

2(l-«2) 


3D 

2^ — lifip 


4 


4A = 1A 




4B-^1B 








4C=1C 


4D = 1D 


5 




5B = 


-(1A)(2B)- 


-(1B)2 


5C = 


(1A)(2C)-(1B)(1C) 


SD= (1A)(2D)-(1B)(1D) 


6 






[(1A)(3C)- 


6C = 
-(1C)2]SB 


-(5C)2 


[(1A)(3D)- 


6D = 
-(1C)(1D)]5B-(5D)(5C) 



7 Z 
Y 



6D 
6C 
J_ 
5B 
1 



[5D-(5C)(Z)] 



X = y^ [ID-dO (Z)-(IB) (Y)] 



cameras. With the Mitchell, if your film is read on a Recor- 
dak with about three special curves on it, the refraction 
corrections could not be entered, but most of the instru- 
mental corrections could be entered. I have never handled 
an Askania, but I would think there might be some possi- 
bility of this. 

In some circumstances might not it be desirable to have 
a computing procedure that would reject the worst of the 
five theodolites on each point and take the least squares 
solution of the other four, or reject the worst of three and 
keep the other two ? 

Mr. Rich: We have the problem at Inyokern of having 
a three-station reduction system. Essentially, we obtain 
checks on the accuracy of the three stations used before we 
even place it into a data reduction scheme. 



Dr. Lotkin: At Aberdeen we are doing exactly the same 
type of problem, among others. As far as the method of re- 
duction is concerned, we have found, by comparison, that it 
is better to use the method of minimizing the squares of the 
sides of the triangle involved, rather than the squares of the 
distances from the lines of sight — one reason being that the 
normal equations become more simple, and we are able to 
put in more checks on the computing procedure as we go 
along. We have been able to mechanize the whole procedure 
on the IBM relay calculators where we start with the 
smoothing operation on the angles, then, by means of least 
squares, compute average position, smooth the position by 
means of least squares again, and obtain velocities and ac- 
celeration. It takes about two minutes per point for this 
whole process; and trajectories containing as many as 300 
points may be reduced within an hour and a half to two 
hours. 



Nlinimum Volume Calculations with M.any Operations 
on the IBM. Type 604 Electronic Calculating Punch 



WILLIAM D. BELL 

Telecomputing Corporation 



PROBLEMS involving a small card volume and many 
mathematical operations or steps are particularly trouble- 
some to handle on standard IBM machines. Usually the job 
will not justify spending any considerable amount of time 
on procedure. If simple machine operations are to be used, 
the number of control panels and associated wiring titne 
become large. A more elegant procedure may consolidate 
the number of control panels, but each panel becomes corre- 
spondingly more difficult, and the total elapsed time may be 
increased rather than reduced. When IBM machines are 
used for general purpose computing, it becomes mandatory 
that some efficient method of handling this type of problem 
be devised. This paper describes one such method. 

Previous attempts to develop an adequate answer to the 
problem discussed here did not meet with much success. 
The advent of the IBM Type 604 Electronic Calculating 
Punch has changed this picture materially. The most effi- 
cient method of solution would be with a card-programmed 
electronic calculator. The method described is admittedly 
crude and slow in comparison with results that can be 
achieved easily with the sequence machine. However, for 
persons having a 604, but lacking more elaborate machines, 
the method to be described will make feasible machine solu- 
tion of many problems which would otherwise be rejected 
as not amenable to punched card techniques. 

Ideally, a procedure utilizing only a single 604 setup and 
card form was desired, which would be capable of handling 
any problem regardless of the number of terms, size of 
amounts and the formulas involved. Within practical limits 
this has been achieved. 

A master deck containing one card for every term, either 
given or to be computed, is used. This deck is expanded by 
reproduction for the number of particular solutions desired. 
Each card contains instructions for the 604. Since a single 
card (or term) may be used in several calculating steps, 
more than one set of instruction fields may be provided in 
each card. 

The 604 is completely under the control of each card as 
it is read. A multiplicity of operations can be called out by 
the card. 



The procedure is simple. A check list instructs the oper- 
ator as to the terms to be used, the sequence of the cards 
and the control field to be used for selecting instructions. 
All the cards are filed by term under index cards. The oper- 
ator selects the proper cards, sorts and runs them through 
the 604. The cards are then separated, refiled, and those 
cards to be used for the next step selected. The process is 
explained in more detail below. 

604 Operation 

The functional control of the 604 is accomplished by 
means of coded punching in a five-column field in each card. 
There can be a maximum of five such control fields in a 
single card. They are labeled A,B, C,D and B for identifi- 
cation. Field selection for the proper control field is accom- 
plished by picking up punch selectors with a rotary switch 
connected to the control panel. The field is selected by the 
operator setting the selector switch at the beginning of each 
operation. 

The operations which can be called out by the card are : 

1. Multiply 2 eight-digit numbers. 

2. Multiply a five-digit constant by an eight-digit multi- 
plicand. 

3. Divide 2 eight-digit numbers for an eight-digit ratio. 

4. Compute sums involving either addition or subtrac- 
tion. 

5. Compute first differences or first sums. 

6. Selective read in. 

7. Selective punching. 

8. A shifting arrangement to select 8 digits to be punched 
from the 13 digits of the electronic counter. 

The versatility which can be attained with these opera- 
tions is surprising. The 604 used had 40 program steps. 
A 60-program machine would make even more flexibility 
possible. 

Card Form 

The card form is shown in Figure 1. Each card is iden- 
tified by the necessary parameters, and by a term number. 
There is a single eight-digit amount field of three whole 
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lOB No. 



TERM 



AMOUNT 



+ 



+ 



K 
FACTOR 



+ 



-Cnoi •£-^□0) -it-CnO) .Jr-CnO) • ^ -^ O <D -it 

-2-1 Q--5^-9--§ tt.-^~_B-| Q--gl^t Q-5l-^-§ °--gl 



0000 

12 3 4 
1111 

2222 
3333 
4444 
5555 
6666 
7777 



9999 

12 3 4 



000 

6 7 8 
1 1 1 

222 
333 
444 
555 
666 
777 



999 

6 7 8 



OjO 

11 12 13|14 

1 1 i!i 



2 2 2[2 

I 

3 3 313 

I 

4 4 4|4 

I 

5 5 5|5 

6 6 616 



7 7 717 



9 9 9|9 

11 12 13114 



OOOiOOOOO 

16 17 t8|l9 28 21 22 23 

1 1 1|1 1 1 1 1 

2 2 212222 2 

I 

333I33333 

I 
4 44144 44 4 

I 
555I55555 



6 6 6i6 6 6 6 6 

777I77777 

888|88 888 

I 
9 9 9|99 99 9 

16 17 18119 20 2122 23 
X 



00 

25 26 
1 1 

22 
3 3 
44 
55 
66 
77 



00000 

28 29 30 31 32 
11111 

22222 
33333 
44444 
55555 
66666 
77777 



9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 

24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 5! 



Figure; 1 



and five-decimal numbers. Associated with it is a power of 
10 field which locates the decimal point. This power of 10 
is not calculated, but is punched into the initial setup cards 
in advance, as determined by the range which a particular 
term may have. ( It is desirable to keep all amounts as close 
to one whole number as possible.) 

The K field permits a five-digit multiplying or adding 
constant to be set up. 

The operational control of five columns has the following 
functional operation : 
Column I. Alpha 

An X will cause the amount to enter Factor Storage 
1 and 2. 
Column 2. Beta 

An X will cause the amount to enter Factor Storage 
3 and 4. 
Colum-n J. Operation 

This is coded. Punching one or more digits will call 
out the associated function. 
Code Function 

1. a -/S 

2. Ka 

3. a-^^ 

4. +« 

5. — a 

6. otjv — ajv-i 

Column 4. Punch Suppress 

An X will prevent punching and readout and reset of 



the electronic counter. The results are always punched 
from the electronic counter on NX cards, which also 
clears the counter. 
Column 5. Shift 

A coded punch of 1 through 5 selects eight positions 
from the 13-place electronic counter for punching. 

Program Sheet 

Figure 2 shows the form of the layout sheet for setting 
up problems. This can be filled out quickly and simply. A 
card is key punched for each line. This becomes a master 
deck which is expanded by the number of solutions. The 
initial data are put in the proper term cards. Then the solu- 
tion is attained by repeatedly running selected groups of 
cards through the 604 as explained above. 

As an example of the relative efficiency, a single prob- 
lem, involving approximately 100 operations for each vari- 
able, required one week for a manual computer. Five simi- 
lar problems were solved using the described method in 
eight hours. 

DISCUSSION 

Mr. Clark: As I understand it, you must have what is 
called a semifloating decimal. You float it when necessary, 
and the rest of the time you leave it fixed. Do I understand, 
then, that when you expect a crossfoot, you must arrange 
your powers of 10 so that alpha and beta must have the 
same power of 10? Is that part of your planning? 
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Constant 


Term 


Power 


A 


B 


C 


D 


B 


. Co 

O c^ Co 


. Co 

<x B t ^- $. 

O Oh Co 




<x 


■to 
oper. 
Pch. S. 
Shift 


a 


to 
Oper. 
Pch. S. 
Shift 




1.0 





X X 


X X 


X X 0|0 


X X 





X X 




2.0 





0X0X0 


X X 


X X 0|X 


5X0 






3.0 


+2 




X X 










4.0 





X X 


X 4 X 










5.0 





0X0X0 


0X0X0 


0X0X0 








6.0 





X X 












7.0 


+2 


0X0X0 












8.0 


+ 1 


10 2 


0X0X0 


X X 


X 


0X0 


X 


5X0 




9.1 





X X 


7 3 








000.50 


9.9 





2 3 


0X0X0 


X X 


X 


0X0 


X 


0X0 




10.0 





X X 


10 3 


X X 








11.1 









6 3 


X 


4X0 






11.2 









0X0X0 





3 






11.9 





X X 




10 3 




X 


0X0 




12.1 


+3 




10 3 


0X0X0 








12.9 


+3 


X X 




10 3 








13.0 


+3 


6 3 









X X 






14.0 


+ 3 











10 3 


X 


0X0 




15.0 


-1 


X X 


X 4 X 


3 4 


X 


5X0 






16.0 


-1 


3 4 


X 5 X 


X 4 X 


X 


0X0 


X 


5X0 




17.1 







3 


X 


0X0 






17.2 







X X 





3 3 






17.9 


-1 


X X 


3 4 


X 1 X o|x 


5X0 






20.0 


-1 


0X1X0 


X 4 X 


3 





X X 





X X 





Figure: 2 



Mr. Bell: The process used in the method described is 
very simple. We have a field which, in size, is three whole 
numbers and five decimals. We multiply that by another 
number with three numbers and five decimals. That is the 
size of the field punched in the card with the decimal point 
indicated, and we obtain a product consisting of six whole 
numbers and seven decimals. That is the way in which the 
604 is programmed to obtain a product. Now we are going 
to keep eight positions out of the possible 13, and shift the 
decimal point as previously determined when we laid out 
the problem. Since the decimal point has been shifted, the 
corresponding power of 10 must be indicated. This indica- 
tion is actually punched into the card when the problem is 
set up, and it is this punching which controls the decimal 
shift. 

Mr. Clark: Don't you look ahead and fix your shift in- 
structions so that the power of 10 will align with the two 
numbers which you expect to add ? 



Mr. Bell: Addition is the problem, then ? Suppose I want 
to add two numbers together and obtain a sum. If the 
power of the two is the same, there is no problem ; but as- 
sume they are different. Suppose the power of the first 
number is 1, and the second is zero. There are several pos- 
sibilities. I could punch a 10 into the zero power card. Then 
I could multiply the first factor by this term and add it to 
the other number whose power was 1. This gives a result 
whose power is 1. 

Mr. Clark: If you are never going to use that factor for 
anything else, you would be just as well off to store it origi- 
nally the way you expect to use it. You might use it other- 
wise ; therefore, it might be an advantage to have it stored 
this way. 

Mr. Bell: There are a lot of modifications with this 
method. The more you use it, the more you see easy ways 
of doing something which was done the hard way first. 

Mr. Belser: Do you keep eight places at all times, no 
matter how many figures ? 
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Mr. Bell: No, we don't. There is nothing in this method 
which guarantees that you will have eight significant fig- 
ures. We designed it primarily for engineering problems 
where a high degree of accuracy and number of significant 
figures were unnecessary. So, in laying out the problem, it 
is necessary to keep track of the size of the numbers, and, 
when the number of digits of the result decreases, shift the 
numbers over and increase the number of digits in the 
answer. 

Dr. Grosch: In other words, this is not a floating decimal 
system, and it is exactly identical to the ordinary work that 
we do in shifting wires on a control panel, except that it is 
useful for small-scale work, since only one control panel is 
used. 

Mr. Bell: Right. 

Mr. Ferher: It is really not limited to just small jobs, 
however. You can do a job with n operations horizontally. 



Mr. Keast: There are many problems where sines, co- 
sines, and logarithms are involved, very frequently. How 
do you feel about that, please ? 

Mr. Bell: Well, in this particular problem a point was 
reached at which square roots were needed. We have a 604 
control panel that evaluates square roots by the iterative 
approach. The 604 actually computes the first guess that is 
used. So we are not forced to use the method discussed. 
We simply take the cards of the values for which we want 
square roots, run them through the 604 with the square- 
root control panel. Then reproduce these cards into the 
field which is to contain the value to be used in repeating 
the calculations. Practically, you could do this iteration 
process on the described setup, but you would use a great 
many cards, and the process would take a long time. 



Transition from Problem to Card Program 



GREGORY J. TOBEN 

Northrop Aircraft, Incorporated 



I WANT to talk about card programming as applied to 
engineering problems. The IBM Card- Programmed Elec- 
tronic Calculator is only now being delivered ; so I should 
explain how we have been able to do some card pro- 
gramming on other combinations. I shall describe a simple 
two-card program for generating sine and cosine, a 13-card 
program for wing loads on an airplane, and a five-card pro- 
gram for a Monte Carlo calculation. Then I should like to 
mention briefly a number of setups made by members of 
our group and interesting program variations in each. Then 
I shall summarize some of the results gained in solving 
these problems. 

In the days prior to the IBM Type 602 Calculating 
Punch, we were presented with a problem involving the 
step-by-step solution of a set of 14 simultaneous nonlinear 
differential equations. No analytical solution was possible 
because of step functions. The two large-scale digital ma- 
chines capable of doing the problem were busy on higher 
priority work, and the problem was not suited for analogue 
computers such as the differential analyzer because of a 
large spread in the size of the numbers. Shifts of 10* some- 
times meant that a gear would have to turn for years to 
effect the result. Moreover, it was a design problem, and 
the whole course of investigation depended upon the early 
results. The solution was important to the guided missile 
program, and we were able to persuade IBM to convert our 
IBM Type 405 Alphabetical Accounting Machine into 
something suitable for the job. They made available an 
IBM Type 603 Calculating Punch, which was then out of 
production, and connected it, via cable, to the 405. Forty 
class selectors and 40 x distributors were added to complete 
the job. The elapsed time from preliminary design to de- 
livery of the machine was only six weeks, and it was so 
well done that after two years of use we have been able to 
think of only minor improvements. 

When the two machines are attached, the 405 provides 
storage space for constants and factors which may be trans- 
ferred from counter to counter to obtain desired combina- 
tions and from counter to multiplicand or multiplier of the 
603 for multiplication. Products may be sent to multiplier 
and multiplicand for remultiplication, raising to power, etc.. 



to the 405 counters for accumulation, or to type bars for 
printing of final results. The sequence of operation is con- 
trolled by a set of program cards fed through the 405. These 
cards contain x or digit punchings to pickup selectors to call 
for the required transfers, etc. These cards may also contain 
factors to be used in computations. The results may be 
listed step by step until the problem checks and then tabu- 
lated and summary punched as usual. When so used, of 
course, the multiplication rate is 150 cards per minute. 
Counter transfers are accomplished by wiring counter exits 
of the transmitting counter to counter entry of the receiv- 
ing counters, with instructions for either add or subtract. 
No special wiring is necessary to take care of negative 
numbers. If a negative or complement number is standing 
in a counter, it appears in the multiplier as a positive num- 
ber, and the sign of the counter receiving the product is 
reversed. A six-place column shift is placed ahead of the 
multiplier output and controlled by x's. Round off may be 
wired on or off. A negative balance test on the counters is 
available. While only one multiplication can be done on 
each card cycle, other parallel operations are always used. 
Table look-ups can be made, additions, counter-to-counter 
transfers and comparing operations are all possible. 

The 603 is not able to divide, and multiplications larger 
than 6 by 6 had to be done in four passes. It was soon ap- 
parent that even the routine everyday work could be done 
more economically on the combination than any other way. 
This one machine keeps six men busy; we have no other 
multipliers and no regular key punch operator. 

Card programming is merely the use of punched cards to 
control selectors and other functions of the machine. Ac- 
tually, it accomplishes a great deal more than this. It is an 
attempt to replace all of the programming that would other- 
wise be done by the operator, when he handles cards and 
changes control panels, as he gathers together the separate 
parts of a computing routine. When all outside handling is 
eliminated, spectacular time and cost savings can be made. 
Ten-to-one ratios over other tabulating procedures are not 
uncommon. Card programming is used to supplement the 
machine programming available on machines such as the 
IBM Type 604 Electronic Calculating Punch. 
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Illustration of Card Programming 

Sine and cosine functions may be generated together 
using two counters for the functions and a third for x in 
radians. The formulae are : 

sin;i- ^icosx d.v 



cos X =/— sin X dx 

If the interval is dx = 1/1000 the products of dx can be 
obtained by shifting the numbers three places ( Figure 1 ) . 

Card 
No. 8Acos 8Bsin 4Ajt- 



1 Read in 


initial 


1.000000. 




.000000 


.000 


values 








■-—-^1000 




2 




l,*- 


- 


— .001000 


.001 


3 




.999999 


+ 


_^ 999 




4 




1 ^«_ 


— 


— .001999 


.002 


5 




.999998 


+ 


^ 999 




6 






— 


— .002998 


.003 



etc. 
Figure; 1 

By using a selector as an alternator the shift may be accom- 
plished without card punching. Card count adds in 4A on 
alternate cards. This procedure will generate a three-place 
sine and cosine, and it is often used as a part of a program. 
Accuracy can be improved by making dx smaller or by add- 
ing a second order term, or both. 

Consider the problem of determining wing loads for an 
airplane (Figure 2). 

It is necessary to know the total shearing forces on the 
wing. These forces will be the weight and lift. From weight 
measurements and wind tunnel tests it is known just what 
these loads will be for level flight. To obtain the loads for 
other conditions such as pullout, dive or landing, they can 
be multiplied by a factor. The bending moment on the wing 



Lift 




Figure; 2 



will be the shears times their distances. The torque will be 
the weights times the distance from the reference line in the 
wing. In addition to these forces, there are chord shear and 
moment due to accelerations in turning, etc. The formulae 
have the following form : 
shear-beam = S] -\- [3 ® 
moment-beam = [H © + E] © 
torque =[7]©-h[B]©- [£]© 

shear-chord =|1]© + E]®+ MO 
moment-chord =[3© + [6]© + 1121© 
The labels on the factors need not be listed, except to say 
that, in general, the circles represent weights times dis- 
tances, and the boxes are load factors. The circles will re- 
main constant for a good part of the run, while the boxes 
change very often. There are 200 different combinations of 
load factors necessary to cover all reasonable attitudes of 
the plane. The five forces listed above are needed at per- 
haps twenty different points or stations from the tip of the 
wing to the root. The problem consists, therefore, of 13 
products, grouped into five sums, for 20 stations and 200 
conditions (13 X 20 X 200 = 52,000 multiplications). The 
job will take 14 program steps, all alike in that two factors 
will be read in and a product read back. For 20 stations 
there will be 280 program cards. These 280 cards must go 
through the machine once, at the rate of 150 per minute, 
for each set of load factors, i.e., 200 times. The seven load 
factors are contained in a single read-in card, and they re- 
main in the machine while one run of 20 stations is being 
computed. 

If the operator runs the program, 50,000 pairs of factors 
should be punched into cards. This will require handling 
50,000 cards, or 10,000 cards five times. Fifty thousand 
multiplication cycles at 100 per minute must be performed. 
In addition, there will be more than a little gang punching, 
sorting, and collating, plus a run in the accounting machine 
to print the answers. The wear and tear on the operator is, 
obviously, much less where card programming is used. It is 
easy for him to watch the answer columns for abnormally 
high stresses, and a recheck of factors can be started imme- 
diately. There have been occasions where wrong signs were 
picked up on load factors and a few erroneous numbers 
resulted. Re-runs can be made in two minutes. 

MoNTS Caring Cai,cuIvATions 

In this type of problem, part of the program is deter- 
mined by a random choice from a file of random digits. Our 
source is the RAND Corporation's 20,000 card file. Each 
card contains 50 random digits and a serial number. The 
problems we perform usually involve the life history of a 
neutron as it goes through a slab of material. Several 
groups are working these problems with a variety of ap- 
proaches. Many here have worked with a mass sample, i.e., 
a thousand neutrons are taken from impact to impact and 
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when the sample is pretty well depleted, statistics are gath- 
ered. With card programming it is possible to follow the 
course of a single fictitious neutron until it is either cap- 
tured or goes outside the limits of the experiment. When 
this happens, the calculator may be restored to its initial 
conditions and the same routine continued with another 
neutron. When stable curves have been produced, the mate- 
rial is ready to be changed or another experiment tried. 

To set up this problem it is necessary to make a numeri- 
cal analogy of each of the physical events to be considered. 
Counters may be used to store the numbers representing 
the probability that a certain event will occur. P'or example, 
if the probability of capture was 5%, a 95 might be stored 
in a counter. Then, if two-digit random numbers were sub- 
tracted from this counter, it would indicate a negative bal- 
ance 5 times out of 100. If there were four possibilities 
stored in four counters and the probabilities were made ac- 
cumulative from left to right, the leftmost negative sign 
would indicate the process to be used. Similarly, the dimen- 
sions of the slab may be stored in a counter and continu- 
ously compared with the position of the neutron. When the 
prog'ress exceeds the slab thickness, the machine is cleared. 

When neutrons travel through a shielding material, there 
are four possibilities : 

Type of Collision Operation 

capture start over 

elastic light energy change 

elastic heavy no energy change 

inelastic variable energy change 

Energy levels are represented by discrete numbers, using 
for the highest energy and dropping to 9 for the lowest. 
The intervals and probabilities are chosen so that they con- 
firm closely to the physical problem. Needless to say, our 
mathematicians do most of the work on a problem of this 
kind. Not only must they contrive experiments with num- 
bers to fit the actual physical conditions, but they must also 
be alert to see that a truly random sample has been used 
and none of the rules of the game has been inadvertently 
violated (Figure 3). 

When the calculator is started, a set of random digits and 
a mean-free path are picked up. At the start the direction 



can only be forward. A direction at random (cos) is selected. 
A multiplication factor is needed, depending upon the 
energy level (L) and a set of process probabilities which 
also depend on the energy level (00). In the meantime, a 
product of PL cos has been formed and accumulation of 
the progress of the neutron (Z) is begun. Now take a 
total, printing everything, but clearing only Ply cos and the 
random digits. The process, as shown by the status of the 
probability counters, is determined for the succeeding im- 
pact which is now considered. Card 1 reads the previous 
direction to a digit selector so as to make a look-up in a 
table punched in card 2. A new pair of digits and a new 
mean free path, both random, are stored. On card 2 there 
is a choice ; if the probability counters indicate three minus 
signs, the number from the table look-up on card 2 is taken 
and used to select a cosine on card 3. The first table look-up 
also provides a change for the energy counter. In case the 
probability counters indicate one or two minus signs, then 
a random digit is used to select the cosine, and a corre- 
sponding energy change is made. Then the new energy is 
read while the progress is computed ; and the process con- 
tinues until either the forward progress exceeds 10 centi- 
meters or turns negative, which means that the neutron 
has come out of the front face, or until the process indicates 
a capture. 

The five card decks are made in sets of several thousand, 
and the 20,000 random digits are collated in as the runs are 
made. Frequent re- randomizing is done by re-sorting and 
by the use of new random digit columns. 

Stress work has been assigned job numbers since the 
first of the year. They are now up to number 120. Work 
has been done for such groups as aerodynamics, thermo- 
dynamics, guided missile section, wind tunnel, quality con- 
trol, NEPA, etc. We estimate an average of four jobs per 
week. The usual procedure is for someone to bring in both 
mathematics and data. The author stands by while the 
operator wires the control panel and runs through a set of 
check calculations. Then the operator stands in front of the 
machine while the answers are being printed. In a week 
the operator doesn't really become familiar with a problem, 
but the large variety makes the work interesting. 
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Figure 4 



STRKSS EXAMPI^IIS 

Fuselage Balance-Tail Loads. First a weight is assumed. 
Then a table look-up is made for the drag and airload, each 
producing torque. From the known lever arm, the up force 
on the tail to balance the torque is computed. Subtract this 
uplift from the wing lift and repeat three times. After the 
third try, a refined table look-up is used which gives fuse- 
lage and wing components separately. Thus, there are eight 
table look-ups and three iteration cycles. 

Wing Loads. Over 1,000 runs were done, averaging 200 
flight conditions on each plane analyzed. Considerable sav- 
ing on cards is possible on each setup. 

Fuselage Loads. Some 800 runs have been made, 442 on 
a cargo plane which had a large number of possible cargo 
distributions. 

Section Properties. Both wing and fuselage section prop- 
erties have been run, using a newly developed tapered beam 
analysis. Thirty stations with three types of bending per 
station are shown, 94 items of area, 94 programs maximum. 
Except for the tail loads, all stress jobs were run on one set 
of control panels, and by operators having no previous 
experience. 

Wind Tunnel Balance Corrections. Six counters were 
used for storage and six for answers. The program involved 
65 program steps giving first and second order corrections 
to six balance readings. Two lines of six components are 
computed each minute, which is nearly 20 times as fast as 
any previous runs. 

Missile Flight Paths. Most of these jobs are classified, 
but they do involve generation of sine, cosine and arc sine, 



by power series to nine places, which is better than can be 
had from any reasonable sized tables. 

Performance. This involves six table look-ups. It was 
necessary to go in with two values and come out with a 
third. For coarse lattices, it was necessary to use second 
differences. This job took twice as long to do with card 
programming as by other methods. 

Conclusion on Programming. A planning sheet should be 
made for each job. A good planning sheet will account for 
75 per cent of the work on most jobs. The form doesn't 
seem to be too important as long as it is consistent. Stand- 
ard or fixed control panels can be used if the work is not 
too varied. Thus, it is possible to program a large variety 
of jobs by changing only the program deck. 

Each program step should do a multiplication ; parallel 
operations should be used whenever possible. 

A sequence control on each program deck should be used 
to insure that no card gets out of step. This is provided by 
punching the number of each card and number of the card 
following (Figure 4). 



DISCUSSION 

Dr. Polachek: This question is not necessarily directed to 
the speaker but to any of these numerous random mathe- 
maticians. The question is about the table of random digits. 
Apparently they have serial numbers. My question is : Sup- 
pose you sort the table on the random numbers ; would the 
serial numbers then become a random table? 

Dr. Hamming: We were required one time to randomize 
the numbers 1 to 1,000; so we took the first thousand cards, 
sorted on ten random columns, one after another, and listed 
the original column. This gives us the numbers 1 to 1,000 — 
and fairly random. 

Professor Tukey: Maybe it needs to be pointed out that 
these are pasteurized random numbers and not certified, if 
you do them this way, because they are going to come up 
without repetition, and in a thousand random numbers 
there ought to be some repetitions ; but for many purposes 
this might be better. 

Dr. Hamming: As you recall, it was to get distinct the 
numbers 1 to 1,000 random without any duplications. 

Professor Tukey: That is what you wanted for that pur- 
pose ; but an ordinary random number table is supposed to 
have some repetitions. This is a different kind of table. 



Besf Starting Values for an Iterative 
Process of Taking Roots'^ 

PRESTON C. HAMMER 

University of California 
Los Alamos Scientific Laboratory 



WITH THE ADVENT of computing machines such 
as IBM type 602, 602- A, and 604 calculating punches ca- 
pable of repeating calculation sub-programs making use 
only of temporary or internal storage capacity, iterative 
processes of calculating certain functions have become feasi- 
ble. The most important such function, in terms of its fre- 
quency of occurrence, is the square root. The question of 
how to pick the best starting values for one type of iteration 
for square roots is answered completely in this note when 
one is concerned with limits on absolute or relative errors 
for a minimum fixed number of iterations. 

Errors in an Iterative Method jar Square Roots 

The iterative method here discussed is the classical one 
given by the formula : 

where x^ is the radicand, a^, k = 1,2, . . . , is the ^th ap- 
proximation to the square root x, and a^ is the starting 
value. The error aT^ — x is readily calculated* by observing: 
(afc_i - x)- 



«A- 



-.r = 1/2 

{ajc-^-\- xY ^ 

From which, by division and extension, we have 



afc+A' =1/2 



(1) 



au — x 






ak-\-x 

Solving (2) for a^ and then for a^ — x in terms of a^ and x 
we have 

x[(a,-{-x)^^ + {a^-xy^] 
{ao-\-xy^ ~ (a^-xY^ 
2x{aQ — x)^^ 



die 



ajc-x ■■ 

ttk — X 



2(a,-x)^^ 
X (ao+x)^^ - {ao—x)2^ 
Now, we observe that for ^ = 1,2, 



(3) 
(4) 



• = 1,2,... . (5) 
the error always 



has the positive sign. It is also readily proved that the error 

*This paper was presented by title. Work was done under U. S. 

government contract. 
^This method appears in Whittaker and Robinson, The Calculus of 

Observations. 



and the relative error as given by (5) are monotonic de- 
creasing functions of a^ for fixed x for a^ <C x and mon- 
otonic increasing functions of a^ for a,, > x. Similar 
monotonic properties hold for the errors as functions of x 
when flo is fixed. 

The Best Starting Values 

We now state the problems : Given a radicand in the 
interval (Nj^,N2) xf = N-^^, xl = N2; to find a single start- 
ing value tto which for a will minimize the maximum abso- 
lute error in the ^th approximation and find likewise a 
starting value which will minimize the maximum relative 
error for all choices of radicands in the interval. 

To solve these problems, we first define an auxiliary 
function 

P{a,x,k) = {a-x)2^/(a-\-x)2^ ,then 
2x P(ao,x,k) 



ajc—x 
au—x 



1 — P(a^,x,k) 
2P(a^,x,k) 



and (6) 

X 1 - P{a^,x,k) ' ^^^ 

From the monotonic properties of the errors as functions 
of the starting values we may state that for any starting 
value a^ the largest absolute error will occur for ^ = .ar^ or 
X = X2. Hence, the largest error will be minimized by 
equating the errors at the upper and lower extremes of the 
range of the radicand. The same statement will apply to 
relative errors. Hence, we have the theorems : 

The;ore;m 1. The starting value a^ which minimises the 
maximum absolute error in the nth approximation for a 
range of radicands between N^ — x^ and N^ = xl is the 
solution of the equation 

2x^ P(ao,x^,n) _ Ix^ P (ao,X2,n) ,^. 

1 — P(ao,x^,n) 1 — P(ao,X2,n) ' 

In general, these solutions a^ depend on the number of 
iterations n, and a^ is a decreasing function of n. The quan- 
tity on the left in equation (8) gives the actual maximum 
error when the solution a^ is substituted. 

The;orem 2. The starting value a^ which minimises the 
maximum relative error for a range of radicands between 
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A^i = xl and N2 = ;f| is a^^ = V-^1^2 = V-^i-*'2^ inde- 
pendent of the number of iterations k. The maximum rela- 
tive error then is 



2 P(\/x^X2,x^,k) 2 P('\/x^X2,X2,k) 



(9) 

l-P{^/X^X2,X„k) l-P(^/X^X2,X2>k) 

It is easily seen that the ratio a^/x^ depends only on the 
ratio X2/X-1 in both theorems ; hence, one may multiply 
aQ,Xi,X2 by a positive number and still have theorems 1 and 
2 valid for the new quantities. The maximum relative error 
does not change under a scalar change. 

To give an idea of the maximum relative errors in using 
VIO as a starting value for radicands between 1 and 100, 
we have that the third approximation gives a relative error 
less than 1.08 X 10"-, the fourth approximation a relative 
error less than 5.7 X 10~"\ the fifth approximation a relative 
error less than 1.5 X 10~®, and the sixth approximation 
a relative error less than 1.26 X lO"'^^. The best integer 
starting value for radicands between 1 and 100 for six 
iterations is 3 for either absolute error or relative error. 
If the absolute error is to be minimized, the best starting 
value for one iteration is (^1 -\- x^ /2. As the number of 
iterations increase, the best starting value decreases to- 
ward y x^X2, although we have not succeeded in showing 
actual convergence to V-*"i-*'2- 

If the machine under consideration can discriminate 
among several classes of radicands, then one can use the 
method here proposed to determine the starting value as- 
sociated with each class. 

High Order Roots 

If one uses the iteration for wth roots 



ttk 



_ 1 /.y" 

~ n \atl 



+ (w-1) afc. 



■)■ 



2,3,... (10) 



the formulas for errors are not simply derived. However, to 
minimize the maximum relative error one has the following 
theorem : 

Thi;ore;m 3. The single starting value for taking nth 
roots which minimize the maximum relative error for all 
radicands between N-^ — x\ and N2 — x\ is 



_ " / x^x^ {xV; - xV) 
'''-^{x,-x,){n-\) 



(11) 



regardless of the number of iterations used. 

Proof: Let a be any positive number. Consider the rela- 
tive error in the first approximation a^ resulting from using 
a as a "guess" for the wth root oi N = jr". This relative 
error is 

a^ — x x^ -\- {n — \)a^ — n a^^ x 



^[(r^^"-)©--] 



(12) 



We want to show that this relative error is always positive 
unless a = X, when it is zero. Let 3; = x/a and denote the 
relative error by z, then (12) becomes 
y^ -{- {n—\) — n y 



n z = 



y 



(13) 



This relative error z is positive, zero, or negative accord- 
ing as the numerator, y" -f- (w — 1) — n y,is positive, zero, 
or negative. Setting the first derivative of this function 
equal to zero, we have 

n y"^^ — n = whence l3;l = 1 . (14) 

The second derivative is 

«(«— 1)3;"-2 (15) 

which is positive for y > 0. 

Hence as .s = for y = 1 is a minimum, and the first de- 
rivative of the numerator is negative for 0<3;<l(;r<a), 
but positive for 3; > 1, we conclude that ^r = if, and only 
if, a = X, and it is otherwise positive. Furthermore, it is 
easily established that z is a mono tonic increasing function 
of I a — .r| for either x fixed and a varying, or a fixed and x 
varying. 

In view of the above, one may proceed as with the square 
roots. For radicands between Nj^ = x^ and A^2 = ■*"2 ^^J 
starting value a will give a maximum relative error in the 
first approximation for either A^^ or N2 by the monotonic 
property mentioned above. Hence, the minimum largest 
relative error in the first approximation will occur when 
the error using AT^ is the same as the error using N2. Equat- 
ing these relative errors, and using the fact proved above 
that the relative error is positive or zero, we have from (12) 



From (16), one finds the unique solution 



fln = 



1 2 V *? 1 / 



(16) 



(17) 



(W-l)(^2— ^1) 

Now we have still to prove that this same value a^ is the 
best starting value, regardless of the number of iterations. 
Before proving that, we remark that the best starting 
value flo to minimize the maximum absolute error with one 
approximation is 



n-i / ^n 
^ n( X 



r \. (18) 

n{x2—x^) 

It may be demonstrated, although we here omit the de- 
tails of proof, that the best starting value a^, to minimize 
the maximum relative error in k iterations, is that value of 
a^ which yields approximations to V-*"i and V-*"" at the ^th 
iterate such that the relative errors in both are equal. The 
following lemma then shows that a^ of (17) is the best 
starting value to minimize the maximum relative errors 
independently of the number of iterations. 

Lem,ma i. If the relative errors in using a.^ as an approxi- 
mation of V-*"** and &i as an approximation of Vy'* are 
equal, then the relative errors in ag and &2 are equal, where 
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q^ and &2 ^^^ the iterates, obtained respectively by substi- 
tuting fli for ttjc-i in formula (10) and 3; for x, h-^ for a^-x. 
Proof: We have given that (a^ — x)/x = (&i — y)/y or 
what is equivalent, a^x — hjy. Now 

x-^{^n-\)a\ _^ ^ _ yn^(n-l)b- 



a, = 



n a. 



and bo 



nb'', 



and 



(19) 



I hence, 

as a^^/x = &1/3; we have a^/x = ^a/y? which is equivalent 
to the conclusion of the lemma. 

This proves the lemma. Now, as a^ was chosen so that 
the relative errors of the first approximations to V-^i = ^1 
and to V^2 = ^2 were equal, it follows from Lemma 1 
that all successive approximations to jt^ and Xz will have 
the same relative error at a given iteration. This concludes 
the proof of theorem 3. 

Concluding Remarks 

We have called attention to formulas for the errors in 
the classical iterative method of taking square roots, and 



applied these formulas to determination of the best start- 
ing values to use, if one wishes to obtain a certifiable 
accuracy in the smallest number of steps. While the de- 
termination of errors for higher order roots was not given 
algebraically, this may be done numerically for particular 
circumstances. For example, in taking cube roots with 
iVi = 1, N2 = 1000, the best value for minimizing maxi- 
mum relative errors for a fixed number of iterations is 



10(10^—1) 



= V55 



2(10—1) 

If one chose to use 4 for a starting value, then the maxi- 
mum relative error would occur at ;r^ = 1 as V 55 is a 
dividing point and 4 > V55. 

It should be pointed out that the methods here discussed 
do not apply if one has a machine for which the number 
of iterations need not be fixed, but which will test the ac- 
curacy at each iteration. In that case, a choice of a best 
starting value would be better determined by minimizing 
the average length of time of calculations for the distribu- 
tion of radicands at hand and for the accuracy desired. 
However, the results stated here will be of some use in 
proceeding with the minimum calculation time determi- 
nation. 



Improvement in the Convergence of IS/iethods 
of Successive Approximation^ 

L. RICHARD TURNER 

Lewis Flight Propulsion Laboratory, NACA 



THE METHOD of successive approximation is fre- 
quently used in numerical mathematics, but in some cases 
the rate of convergence is discouragingly slow. Professor 
Southwell has shown that the rate of convergence of Liep- 
mann's method of solving partial differential equations may 
be substantially improved by examining the distribution of 
"residuals" (increments in Liepmann's method) and apply- 
ing local corrections. 

Southwell's "relaxation" technique is not readily adapt- 
able to machine computing methods. It is possible, however, 
by examining the whole solution to determine the rate of 
disappearance of the currently dominant error terms and 
then to remove such dominant terms in a single step of 
calculation. 

Theory of the Method 

Let the ultimate solution of a given problem be the 
i^-dimensional vector x : 

A = \Xi, X2, X^, • ' • ) ^k) ) 

which is obtained as the limit of the convergent sequence 
X = lim^X(o>,X<i>,X<2), ...,X<»>,....}- . (1) 

(It will be assumed in the analysis that the components 
^(m) q£ ^j^g yy^^ iteration are all real numbers, although this 
restriction can be removed.) 

We now suppose that, at the wth step, that X^"^ is com- 
posed principally of the solution Zand two error terms B^"^ 
and i^^**^ of a form such that 

and /^<"^'> = — aF^"> . (2) 

Then it is found that 

X(») =X -\- £(«> + i^f''> 

X(«-i) = X -f aH<»> - kP^''^ 

Xin*^) = X -f a2H<»> -I- K^P^''^ 

X(«^3) = X + A^B^'*^ - PF("> (3) 

*This paper was presented by title. 



SO that 



and 



X' = 



X<"+3) J^(n+2) 

X<«+3) A^X'"'^^^ 



(4) 



(5) 



1-A^ 

Now, in general, the operation indicated in equation (4) 
is not even defined. Therefore, some adequate working ap- 
proximation must be substituted for equation (4). Two of 
these appear to be worthwhile. We define 

S^A^i) = ;ir(.-s^+i) —x^.'^^ (6) 

) } } ^ ' 

and in terms of these 8's which are defined for each point 
for which a calculation is made 






§(n+3) g(n+i) 
} } 



/isr 



A? 



E 



}=\ 



j 



(7) 



or 



A 

E 



g(m+3) g(n+i) 
3 i 



A 



2 J=i 



(8) 



g(n+i) g(n+i) 
} J 



3=1 



Equation (7) is meaningful only if the 8's are real numbers. 
Equation (8) makes sense for any definition of Sj for which 
a complex conjugate Sj and the operation ab are defined. 
Equation (8), which corresponds to taking a first moment 
in statistics, is more elegant than equation (7) but involves 
much more effort [and is really not much better, because it 
is only on rare occasions that the initial hypothesis, equa- 
tion (2), is sufficiently near to the truth to justify the use 
of great precision in the adjustment indicated in equation 
( 5 ) ] . For this reason it is recommended that, where at all 
possible, equation (7) be used. This rule should not be 
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applied if i^ is a small number. In this case equation (8) is 
a much safer rule. 

When X^ has been found, equation (5) is applied to each 
of the elements of X^"+^^ and X^^*'^'> to obtain an improved 
iterant X'. 

Application Notes 

Strictly speaking, the basic hypothesis of the method can 
be met only for linear algorithms, that is, algorithms in 
which the n -{- 1st iterant is the result of linear operations 
on the elements of the wth iterant. In practice the method is 
found to apply satisfactorily to various nonlinear processes 
such as the calculation of the latent roots of matrices by 
repeated multiplication and normalization of an arbitrary 
vector. 



It is suggested that if 1^ is a small number, say 0.1, the 
correction technique should not be applied unless it is found 
that A- is substantially constant for several iterations. 

In the use of the method by the author, no case has oc- 
curred in which Xr fell outside the range to 1. Such cases 
will form a fresh field for the experimentalist. 

Illustrations 

Three charts illustrate the method for the solution of 
Laplace's equation, with the boundary conditions shown 
around the edge of the L-shaped domain of the figures. The 
initial approximation (Figure 1a) was taken to be zero at 
all interior points. The (n + l)st iteration w = to 2 was 
computed as , 



^. 



(»+i) 



5[' 






(»)i ^ (») _j_ ^.(n) 

T -^ij-l ~T'* 1,1+1 • 
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Figure: 1a. Rksults oi^ the First Three Normal 
Iterations and First Accei^Eration 
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Figure 1b. Results oe the Second Three Normal 
Iterations and Second Acceleration 
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Three iterations were carried out on each sheet, after which 
K' was computed from equation (7) and the underscored 
result x'ij computed. The corresponding values of X- and 
aV(1— X^) are: 

Figure X^ XV(1— X^ 



lA 


0.422 


0.731 


1b 


0.337 


0.509 


Ic 


0.478 


0.915 



The "errors" are the sum for all points of the absolute 
value of the deviation of each .jr^"? from the true solution Xij. 
The "reduction" is the reduction in the error from its value 
in the last preceding iteration. 



The final error (Figure Ic) after nine normal iterations 
and three acceleration adjustments is approximately one- 
tenth of the error that would result from 12 normal itera- 
tions. A similar reduction of error without the use of the 
acceleration technique would have required roughly 20-25 
iterations. 

The acceleration technique has recently been applied suc- 
cessfully to the calculation of the latent roots of matrices. 
In this case the adjustment is made on the value of the ele- 
ments in the characteristic vector or modal column. Since, 
as a rule, only a few elements occur in the modal column, 
it is recommended that equation (8) be used to compute 
X- in such cases. 
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Figure; Ic. Resui^ts oi^ the Third Thrive: NormaIv 
Iterations and Third Acce;i,eration 



Single Order deduction of a Complex Nlatrix 
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RANDALL E. PORTER 

Boeing Airplane Company 



THE REDUCTION of a matrix is a systematic appli- 
cation of the rule that the vakie of a matrix is not altered if 
each element of a row (or column) is multiplied by the 
same number, and the products subtracted from (or added 
to) the corresponding elements of another row (or column) . 
This rule is used to reduce all but one element in a column 
(or row) of a matrix to zero. 

Repeated reductions may be employed to alter the form 
of the matrix to one from which a desired result may be 
readily obtained. The determinant of a square matrix, 
for example, may be obtained from the "w fold" product 
of the main diagonal of the equivalent triangular matrix 
(Figure 1). 



Original Matrix 
X X X X 



X X 

X X 

X X 

X X 

X X 



X 
XX X 
XXX 
XXX 
XXX 
XXX 

D = Product of These Terms 



•^Equivalent Triangular Matrix 

^x 

x^x 

X x^x 

X X x^x 

X XX X X 

X X X X X X 



Figure 1 

The solution of linear simultaneous equations, the compu- 
tation of a reciprocal matrix, etc., may be accomplished by 
using a reduction procedure to alter the form of the matrix. 
The following reduction procedure is for use with mat- 
rices having complex elements. No expansion of the matrix 
elements is required. A standard IBM Type 602-A Calcu- 
lating Punch may be used for the required calculations. 

Key Punch Instructions 

Punch one card for each element of the matrix as indi- 
cated below: 

Item Max. Size Card Cols. 

problem no. xxxxx 1-5 

row no. XX 6-7 

col. no. XX 8-9 

*This paper was presented by title. 



Item 


Max. Size 


Card Col 


*real sign 


X 


10 


real amount 


XXX . xxxxx 


11-18 


*imag. sign 


X 


19 


imag. amount 


XXX . xxxxx 


20-27 


common x-28 







*NOTE: Punch "0" for plus and "x" for minus. 

Check key punched cards by listing and auditing, or by 
verifying. 

Card Preparation Instructions 

1 . Reproduce working cards from key punched cards as 
follows : 



Item 
problem no. 

k 

row no. 
col. no. 



* 



**real amount 

**imag. amount 
common "x" 



Max. Size 
XXXXX 

XX 

XX 

db 
Oxxx . xxxxx 

± 

Oxxx . xxxxx 

X 



Prom Into 

cols. 1-5 cols. 1-5 

and cols. 80-76 
cols. 6-7 cols. 41-42 

and cols. 40-39 
cols. 8-9 cols. 43-44 

and cols. 38-37 
cols. 10-18 cols. 45-53 



cols. 19-27 
col. 28 



cols. 54-62 
col. 7 



*NOTE : Punch field in reverse. 
**NOTE : Punch the algebraic sign over the last position and gang 
punch a "0" in the first position. 

2. Hold the original key punched cards aside. 

3. Select all cards where the row (columns 41-42) 
equals the column (columns 43-44) and reproduce 
80-80, gang punching a common x-44 into the extra 
deck. 

4. Combine all reproduced cards and sort to row ordor 
within the problem. 



Item 


Card Cols. 


Type 


problem no. 


1-5 


major 


row 


41-42 


mmor 



Summarize the cards by row on an accounting ma- 
chine. Group indicate the problem number into col- 
umns 1-5 and columns 80-76, and the row into col- 
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umns 41-42 and columns 40-39 of the common x-7 
summary cards. Punch the summations of real and 
imaginary terms into the summary cards (real 
amount in columns 45-53 ; imaginary amount in col- 
umns 54-62). Hold the detail cards. Summary cards 
are used in step 6. 

6. The summary cards are to be used as check column 
cards. Gang punch a column number "0 0" into col- 
umns 43-44 and 38-37 of all summary cards. 

7. Combine the detail cards from step 5 with the sum- 
mary cards from step 6 to form a complete deck ready 
for reduction. 

Reduction Procedure 

1. Sort all cards in reverse order by row (columns 41- 
42). 

2. Select the last row cards from the front of the pack. 
Hold the unselected rows until step 4. 

3. Pull all x-44 cards from selected row cards, and place 
the x-44 cards with the unselected cards of step 2. 

4. Place the nx-44 cards (last row) from step 3 in front 
of and face to face with the cards from step 2 and the 
x-44 cards from step 3 (top edges up). 

5. Sort all cards to column order within the problem 
number. 

Item Card Cols. 

problem no. 1-5 

column 43-44 

6. Detail x-7 gang punch and master 
(master card is x-7 card which has been reversed by 
step 4) . Gang punch : 



Type 
major 
minor 
x-74 compare 



Item 
*imag. amount 
*real amount 
compare 
Item 
problem no. 
column 
imag. amount 
real amount 



From 
cols. 19-27 
cols. 28-36 



Into 
cols. 19-27 
cols. 28-36 



Against 
cols. 1-5 
cols. 43-44 
cols. 19-27 
cols. 28-36 



From 

cols. 1-5 

cols. 43-44 

cols. 19-27 

cols. 28-36 

*NOTE : These fields are read in reverse order. 

7. Select all x-74 gang punch masters and hold them 
aside. 

Select the last column cards from each problem by 
sorting on column. Hold the unselected column cards 
until step 15. 

Multiply the selected last column cards on a type 
602-A calculating punch wired to compute : 



8. 



9. 



M = ge -\- hf 



Ni = [gf — he] i 



xxxx . xxxxx 
cols. 10 18 

± 



xxxx . : 



cols. 63- 



-71 



10. Reproduce the multiplied cards as follows : 
Item, 
problem no. 

M 
working spots 

Ni 
problem no. 
common "x" 



Into 
cols. 1-5 
cols. 45-53 
cols. 37-44 
cols. 54-62 
cols. 76-80 
col. 9 



11. 
12. 



13. 



From 

cols. 1-5 

cols. 10-18 

cols. 37-44 

cols. 63-71 

cols. 76-80 

col. 7 

Hold the common x-7 cards aside. The common x-9 
cards are now in the original card form except for the 
common "x". 

Select all x-44 cards from the common x-9 cards. 
Place the x-44 cards in front of and face to face (top 
edges up) with the nx-44 cards and sort on problem 
number (columns 1-5). 

Multiply the step 12 cards on a type 602-A calculat- 
ing punch wired to compute : 

± 
xxxx. xxxxx 



^ 



M 



g' + h' 



cols. 10- 



-18 



Ti = 



Ni 



± 
xxxx . xxxxx 
cols. 63 71 



g'-]-h^ 

from the cards with the fields as indicated in Figure 3, 
page 140. 

14. Select the x-72 masters and hold them aside. 

15. Place the x-9 cards ahead of the x-7 detail cards from 
step 8 and sort to row order within the problem. 

Item Card Cols. Type 

problem no. 1-5 major 

row 41-42 minor 



X-7 



Punch Fields 



f 


















M 


hi 


g 


e 


fi 


Ni 





















Gang 
Punched Fields . 



t 



from the cards with the fields as indicated in Figure 2. 



/ 

Card Axis 



Figure; 2 



Normal Fields 
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Punch Fields 
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Multiply the step 15 cards on a type 602- A calculat- 
ing punch wired to compute : 

xxxx . xxxxx 

a' = a — Sc-^ Td cols. 10 -18 

± 
xxxx . xxxxx 

h'i = [b-Sd — Tc] i cols. 63 —71 

from the cards with the fields as indicated in Figure 4. 

Select the x-9 row masters and hold them aside. The 
remaining x-7 detail cards comprise a matrix with all 
values in the last column reduced to zero, except the 
value in the last row when combined with the column 
gang punch masters used in step 6, in the table below. 
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Check Procedure 

The arithmetical operations may be checked as follows : 

1. Select all the column "0 0" check cards from the step 
17, x-7 detail cards. 

2. Tabulate the unselected x-7 cards by row, summing 
the a' and h'i fields separately. 

3. Compare the totals of each row with the a' and h'i 
values in the column "0 0" cards for that same row. 
They must compare. 

4. Errors may be corrected by using the general formula 
given below : 

[a' + h'i] = [a + hi] -[c + di] 1^+^.] 

where : [a + hi] is any complex element in the ma- 
trix outside of the last row or last 
column. 

[c -\- di] is the complex element in the last 
row and the same column as 
[a 4- hi] . 

[e -}- fi] is the complex element in the last 
column and the same row as 
[a + hi] . 

[g -{- hi] is the complex element in the last 
row and the last column. 
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Simplification of Statistical Computations as 
Adapted to a Punched Card Service Bureau^ 

W. T. SOUTHWORTH J. E. BACHELDER 

State College of Washington 



THE punched card service bureau at the State College of 
Washington will be used, in this paper, as an example of 
how the statistical computational needs of a college or uni- 
versity may be obtained as a by-product of an installation 
established primarily for servicing the needs of the admin- 
istrative offices. 

By standardizing and simplifying punched card statistical 
methods, training of instructors and students, and organ- 
izing time schedules, many statistical computations may be 
processed mechanically by the person who will analyze the 
results. 

In almost all of the educational institutions, installations 
are either leased by an individual division of the institution 
or leased as a self-supporting service bureau. In the former, 
it would be a rare case when one division alone could recon- 
cile the cost in maintaining enough of the right type of 
equipment and personnel to service them efficiently. In the 
latter, too much time is wasted in accounting for the time 
spent by the operators on each type of equipment used for a 
scheduled operation in order that appropriate charges may 
be made. This, in institutional work, is "taking money out 
of one pocket and putting it into the other." Most important 
in either case is the loss of the flexibility in using the 
punched card records of one department to develop, assist 
or maintain the records of another department. 

The example installation experimented by giving the di- 
rector a working budget to cover rental, salaries, supplies 
and all incurred expenditures. This budget is reconciled, in 
statement form each year, by the director to the business 
manager, in the form of tangible savings in salaries directly 
due to punched card services and intangibles such as faster 
service, accuracy of records, and services performed that 
could not have been accomplished by manual methods. 

The important point is that the director has complete 
control of all proposed applications, and can use one de- 
partment's records to aid another's. As a result of this or- 
ganization the volume of the punched card application has 

*This paper was presented by title. 



expanded to the extent where up-to-date models of almost 
all types of punched card equipment and experienced per- 
sonnel can be maintained and their costs reconciled. Thus, 
there are few limits, if any, to the types of application they 
can undertake. 

The bureau has the following IBM equipment : 

1 completely equipped alphabetic accounting machine 

80 counters, 88 type bars, automatic carriage, 6 class selec- 
tors, 2 digit selectors, progressive totals, card cycle trans- 
fer device, and net balance. 

1 summary punching reproducer (80-column comparing) 

1 alphabetic interpreter 

1 collator 

1 card counting sorter 

1 electronic sorter (high speed) 

2 alphabetic duplicating key punches 
1 numerical verifier 

Personnel of the bureau is as follows : 

A director to survey new applications, keep procedures up-to- 
date, direct administrative details, instruct new personnel, 
and instruct in courses teaching punched card technique. 

An operator-supervisor to operate equipment and supervise all 
machine work, and furnish guidance to students and in- 
structors processing their own studies. 

One additional operator 

Two key punch operators 

One part-time employee to fill in "peak loads" in either ma- 
chine operation or key punching. 

Work load of the bureau is divided about as follows : 
President's office, for personnel accounting, 5% 
Accounting office, 40% 
All phases of payroll for 3,500 part- and full-time employees 
Budget reports (encumbered) to each department of the 

college and associated financial reports and statements. 
Fee distribution 
Billing of veterans' fees 
Annuity reports 

Student damage claim and deposit statements and checks 
Withholding tax annual statements 
Registrar's office, 25% 
All phases of student records completely converted to 
punched cards including class lists, drop and add control, 
grade reports, posting of permanent records, academic 
standings, and innumerable student statistical reports. 
Statistical work — administrative requirements, 5% 
This covers many varied types of operations which are proc- 
essed completely by the staff of the service bureau. 
Housing and food service, 10% 
4,200 income accounts 
Dormitory listings 
Expense accounting 
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Statistical computations for professors, instructors and grad- 
uate student research, 10% 
This covers guidance time expended, key punching of cards, 
and machine time allowed. The instructors or graduate 
students operate the equipment and process their own 
studies. To accomplish the above, methods had to be estab- 
lished to simplify and standardize the more common sta- 
tistical requirements. The balance of this paper is devoted 
to presenting some of these methods, many of which are 
generally known but seldom used on account of lack of 
cooperation between research and punched card techni- 
cians within the institution. 

The most important simplification of the bureau is the 
instruction given to staff members and graduate students 
who anticipate processing their research project or thesis 
with the aid of punched cards. 

A two-credit-hour course is oflfered each semester, in- 
ckiding one hour of lecture and three hours of evening labo- 
ratory in the service bureau. This course is offered to all 
staff members and students with a prerequisite of one year 
of statistics. It is the purpose of this course, however, to 
attract the persons who will be most likely to have need of 
punched card services in the future. The outline of the 
course is as follows : 

A. The Theory of the Punched Card 

1. The card 

2. The purpose of each machine 

3. Multiple punching 

B. Statistical Applications 

1. Sampling techniques 

2. Construction of schedule or questionnaire 

3. Coding 

4. Data checking 

5. Cross-tabulations 

C. Operation and Wiring of Punched Card Equipment 

1. Key punch and verifier 

2. Sorters 

3. Interpreter 

4. Reproducer 

5. Collator 

6. Alphabetic accounting machine 

7. Brief explanations of the functions of punched card 
calculators, i.e., multiplier, electronic calculator and 
statistical machine. 

D. Statistical Applications and Their Related Machine Pro- 
cedures 

1. Frequency distribution 

2. Matrix distribution 

3. Correlations 

4. Analysis of variants 

5. Chi square 

6. T ratios 

7. Standard deviation 

E. Planning Research Projects to Utilize Punched Card 
Equipment 

1. Hypothesis 

2. Measurement 

3. Format preparation 

4. Coding problems 

5. Punching problems 

6. Analysis 

A professional library of many of the latest books and 
publications on punched card applications is maintained by 
the service bureau and is available to all interested persons 
at the institution. 



The Washington Public Opinion Laboratory at the State 
College has opened its doors to all persons entering into the 
organizational stages of a statistical research study to offer 
advice on sample design interviewing methods, or schedule 
design. This office is proficient in the art of adapting the 
forms used to an efficient document, for transfer to punched 
cards by key punching. As completed data are returned 
from the field they are transferred to punched cards by the 
key punch operators of the bureau. The director or super- 
visor of the bureau then furnishes the necessary technical 
guidance for the student to process his own study mechani- 
cally. 

The; Sum of Products and/or Squares 

The bureau's method in establishing a standard proce- 
dure is to compute all data entering into the many types of 
statistical formulas using ^X, ^X'l, XX^X^, XX^X^, etc. 
The equipment required is : 

1. A permanently wired control panel as illustrated in Figure 1. 

2. A standard sorter. 

3. An 80-counter alphabetical accounting machine. 
For the operating theory see Figure 2, page 144. 



Sample Procedure 

Operating procedure for multiple correlation of 9 three- 
digit variables X-^, X^, through X,,, using 1000 samples 
(raw data formulas will be used). 

1. Inasmuch as there are many methods of creating the cards 
to be used (transferring from another punched card file 
by reproducer, key punching, etc.), we shall start with the 
completed detail cards illustrated in Figure 3, page 145, 
assuming that the data have been checked for a straight 
line of regression. 

2. Sort all detail cards on the units position of each variable 
to reject all missing variables. These must either be ad- 
justed to the average of the others or deleted from the 
study. 

3. Assuming that none of the variables will exceed 450, digit 
cards, consecutively numbered from 000 to 450 in each of 
the 9 variable fields, will be required. Almost every punched 
card installation maintains a consecutively numbered set of 
punched cards for many varied applications. Reproduce the 
consecutive number file to blank cards, transferring the one 
consecutive number field to the 9 variable fields by split 
wiring. A designating x punch is gang punched into column 
80 of the card. This operation is handled by the staff of 
the bureau and will consume no more than fifteen minutes. 

4. Placing the digit cards in front of the detail cards, sort the 
Xi variable field in descending sequence. To sort cards in 
descending sequence, the cards are sorted on the units, tens, 
and hundreds positions the same as for ascending sequence, 
but the cards are removed from the pockets of the sorter in 
reverse sequence (9's in front of 8's, 8's in front of 7's, etc.) . 

5. The permanently wired control panel shown in Figure 1 is 
maintained at all times in the bureau. To this control panel, 
wire variable fields X\, X^, Xs, X4, X5 from the add 
brushes to the Xi, X2, Xs, X4, X5 entry fields of the right- 
hand panel. 



ALPHABETIC ACCOUNTING MACHINE CONTROL PANEL TYPE 405 WITH NET BALANCE COUNTERS AND AUTOMATIC CARRIAGE CONTROLS 
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Figure; 1. Sums oe^ Squares and Products (5 Variabi,e;s) 






SUM OF PRODUCTS AND/OR SQUARES BY CARD CYCLE TOTAL TRANSFER 

In several types of mathematical and statistical calculations it is desired to obtain the 
sum of products or squares. Normally for such calculations first the individual multiplications 
are carried out or the squares are calculated and then their sum is obtained by adding up the 
individual products, although actually only the sum 'of the products is desired. For example 
when an inventory is taken the quantity of each item is multiplied by its unit price and the 
sum of the products is performed to arrive at the total value of the items. In psychological 
tests, when calculating the standard deviation, only the sum of squares has to be obtained. 
For correlation analysis the sum of the squares and products of scores (rating of individuals) 
are needed as 2X,-, 2X, X..., SXi X,,, 2X, Xj, etc.. Several IBM methods are known by which 
the total of products or squares is obtained on Electric Accounting Machines without carry- 
ing out the individual calculations. Such methods, as digiting, progressive digiting and 
digiting without sorting, are also discussed in Pointers. Generally one run through the Ac- 
counting Machine is required for each position of the multiplier for calculating the sum of 
products or squares. The partial totals obtained have to be added up in order to secure the 
final total. 

The sum of squares and or products can be obtained by a single run of the IBM cards 
through the Type 405 Accounting Machine. The cards are sorted together with a set of 
X-punched "digit cards" in descending order (from highest to lowest) on the field of the 
multiplier (or field to be squared) . If the multiplier field consists of one column, 9 digit cards 
(9-1), two columns 99 digit cards, three columns 999 digit cards, etc., must be sorted in. If 
there is no detail card for a number, at least a digit card for that number will be present. 
After the file of cards is sorted, all digit cards preceding the first detail card are removed. 
The last card of the file will be the digit card number "1". The cards are tabulated on a Type 
405 Accounting Machine and totals accumulated by multiplier group are card cycle total trans- 
ferred from one Counter-Group into another to obtain the final total of squares and/or prod- 
ucts. By this procedure diff'erent sums of squares and/or products may be obtained simul- 
taneously. This is especially important for correlation analysis. 

Exhibit A shows a plugging for obtaining the sum of squares of variable X, and the sum 
of products of variables X, X;.. Two Counter-Groups are necessary to obtain each sum. One 
of each pair of Counter-Groups must be equipped with Card Cycle Total Transfer Device . 
Themultiplicand X2, (Columns 29-31)is plugged directly to Counter-Group 8B (plugging 1). 
Columns 26-28 ( as multiplicand ) are plugged to Counter-Group 8A through the NO-X side 
of Selector E in order to prevent accumulating from digit cards (plugging 2). The add and 
subtract impulses are also under control of this Selector (plugging 2*). When a digit card 
passes the upper brushes the X in column 80 sets up Selector E so that when this card passes 
the lower brushes Selector E is in controlled position and accumulation fi'om this digit card 
is eliminated. When a digit card passes the lower brushes Counter-Groups 8A and 8B subtract 
and the totals standing in them are transferred to Counter-Groups 8C and 8D respectively 
which add these totals (plugging 3). Simultaneously, these totals may be listed (plugging 
4). After the transfer, Counter-(3roups 8A and 8B (equipped with Card Cycle Total Transfer 
Device) will contain the same figures that they did before the transfer. If there are no detail 
cards but only digit cards present for a group, the totals transferred for the previous group 
will be transferred again for this group. After all cards have passed through the machine, 
Counter-Group 8C contains the sum of squares (2X,-), and Counter Group 8D the sum of 
products (2X, X.) . These totals are printed as final totals (plugging 5) . The final figures 
standing in Counter-Groups 8A and 8B are the totals of the single items (2X, and 2Xa) 
accumulated from the fields 26-28 and 29-31 respectively. These totals are the last items 
listed by plugging 4. Counts by multiplier group will be obtained in Counter-CJroup 6B and 
total counts in Counter-Group 6D (plugging 6). Counter-Group 6B must also be equipped 
with Card Cycle Total Transfer Device for restoring itself after each transfer. Exhibits B 
and C illustrate an example for this application. 
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CORRELATION OF ENTRANCE TESTS TO GRADE POINT AVERAGE 
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Remove all digit cards preceding the first detail card and 
tabulate on an IBM Type 405 Alphabetical Accounting 
Machine. The following results are obtained : 

a. Count of detail cards used 

b- 2^1 

c. 2^1 

d. 2X1Z2 

e. iXtXs 

f. 2X1^4 

g. S^i^s 

Post these results to the first horizontal line opposite X 
in the correlation chart illustrated in Figure 4, page 146. 

6. Change wires in accounting machine control panel as 
follows : Variable field 6 to control field 2 

" 7 " " " 3 

" 8 " " " 4 

" 9 " " " 5 

Tabulate cards the second time in the same sequence. The 

following results are obtained : 

a. C.C. of detail cards 

b. 2^1 

c. 2^'i 

d. 2^1-^6 

e. 2A^i^7 

f. -^XiXs 

g. 2-YiZ9 

7. Sort column 80, separate digit and detail cards. Place digit 
cards in front of detail cards and sort the variable X2 fi^ 
in descending sequence, the same as the variable Xi field, 
was sorted. \ 

8. Change wires in control panel as follows : 

Variable field 2 to control field 1 
" 3 " " " 2 

" 4 " " " 3 

" 5 " " " 4 
" 6 '\ " " 5 _ 
Remove all digit cards preceding the first detail cards and 
tabulate. The following results are obtained : 
a. Card count of detail cards 



b. 


2^2 


c. 


2^2 


d. 


2^2X3 


e. 


2^2X4 



f. 2^2X5 

g. 2^2X6 

Post these results to the second line of correlation chart. 
9. Change wires as follows : 

Variable field 7 to control field 2 
" 8 " " " 3 

ti u g, « « « 4 

" 1 " " " 5 
Tabulate with cards in same sequence. The following re- 
sults are obtained : 

a. C.C. of detail cards 

b. ^X2 

c. 2^1 

d. 2^^2X7 

e. 2X2^8 

f. 2X2X9 

g. 2X2X1 

Check items 1, 2, 3 to the posting of the second line of cor- 
relation chart. Complete posting of the second line with 
items 4, 5, 6, 7. Check posting of Z2X1 on the second line 
with posting of XxX^ on first line for verification. 
10. Sort, tabulate, and post the results of each variable X3 
through Z9 the same as X^ and X2 to complete table shown 
in Figure 4. A study of this table will readily reveal the 
ease with which the data may be substituted in the various 
multiple correlation formulas by the calculator. No attempt, 
at this institution, has been made to compute the formulas 
themselves by punched cards. 

Advantages of the foregoing methods are : 
1 . Accuracy 

a. All sums of cross products are totaled twice, in 
separate machine operations, with the cards in a 
different sequence and through separate machine 
circuits. There can be no doubt of the accuracy if 
all cross checks {X^X^ against X^X-^, etc.) are in 
balance. 

b. Card counts for each tabulation prove cards are 
all present. 

c. The sums of each variable, which are wired to a 
control field, are produced at the end of each run. 
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CORRELATION TABLP: 





Card 
Count 


%x 


^Z2 


X, 


X, 


Xs 


X, 


X, 


Xe 


X, 


X, 


X, 


X, 


/ooo 


^6345/ 


5^52/S-^3 




74/32/SS£ 


f0S4S2/S2 


4^32/7^/ 


fa342SS^ 


4279Sm 743532^/ 


72743/:afi Sig/ff2/.^3 


X, 


/at^o 


7a£.^3 


74342/5^'^ 


74/fP/^S^ 




^^38^2/S 


4^3/^7S4 


705^09S5 


437So^k 


9SS^fo2/ 9SS32/42I /J47SS32 


X, 


/ooo 


2^^JiS2 


4a^50l3S 


/04^2/Sp 


4d>ViSZ/5 




7S942/S3 


i3^^i22^ 


/^790>432 47^32/57 103038702 7S9S2/7i95 


X, 


MOO 


^7A/7S2. 


7SS43Z/S9 


4£32/7S9 


4l'i/^7^ 


7/if^2/f^ 




74^P/^^t 47^S/Z33 49S2/SS7 m2S^f7^ r/A'S'-^^/O 


Xr, 


/OOO 


4^S7A</ 


es-/^7S/s-/3 


(Af{42S^^ 


yoSAoHifi 


43^.^/27/!/. 


743Z/S^Z 




mS/2S/? ^742/05 ■'^J^A^?^ 42aJS42/ 


X, 


/OOO 


/3S7^/^ 


2/^7/lTP/4 


4Z79^S'^ 


437S/I542. 


/.^79043£: 


47SS/23^ 


473.^/2.^/7 




^oS3/S42/ ?^/S2i77S 4^.J2/4Z5 


Xr 


/OOO 


89432/ 


3^4Z/587S 


/43^P^/ 


fsd^ai/ 


47,^32/S7 


49S2/^'y 


^S742/a6 


2o.':'=i/^4^/ 




9£4Z/£^a3 /^J67<S/S^ 


X, 


/ooo 


7^/?7^ 


/O^iOT/^A 


72/^.^^^ 


1i^i/4:3/ 


4a30iS7!9oi 


/a3Zi/7S9 


4a^2/S7S9. 


3^^^4/7A 


^42/^a3 




3742/S7A 


X,, 


/aao 


f4S7SZ 


f32/5S72/ 


S^^A&z/SS 


f347^3Z 


1^2/7S5 


/7SS43/C/ 


4203^2/ 


4^2/425 


/o^7S/S4 


3742/S76 





Figure; 4 



2. 



This affords many checks for the sum of each 
variable. 

d. The sum of the squares of each variable is totaled 
twice, but each time in the same machine circuit. 
To check the machine accuracy (thoroughly), one 
variable is split-wired to each of the five control 
fields, and a test deck is tabulated. If all results 
are the same, it is a good proof that the machine 
is operating properly. It is recommended that a 
test deck of approximately 200 cards be main- 
tained with the control panel to test the machine, 
before tabulations are started and again when 
completed. 

Speed of Operation. The illustrated problem can be 
completely processed to the completion of the correla- 
tion table in a little less than eight hours. 

The procedure is simple enough in structure that 
persons not trained in punched card techniques can 



process their own studies with a minimum of guid- 
ance. 
4. Each digit card, as it passes through the machine, 
forces the totals standing in each counter to list. This 
automatically produces a cumulative frequency for 
each variable. 

4- and ^-Digit Variables 

Since one digit card is needed for each progression, obvi- 
ously it is not feasible to process 4- or 5-digit variables in 
one run through the machine. A 4-digit field is split into 
two 2-digit fields, a 5-digit field into 2 and 3. The resulting 
squares for each portion are then offset two places and 
added together. 

This method of computing the sums of squares and the 
sums of cross-products has been used successfully at Wash- 
ington State College for over a year. The wiring of the 
permanent control panel and the establishment of standard 
procedures have saved many hours of work. 
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Cross Tabulations 

I am sure that most persons operating punched card in- 
stallations in institutions are familiar with the methods of 
obtaining cross-tabulations through digit selection on the 
alphabetical accounting machine. It seems, however, that 
very few of them are using this method extensively because 
of the amount of time consumed in control panel wiring for 
each individual project or study, It has been proved that 
one permanent control panel could be wired in such a way 
that the changing of five wires would produce 90 per cent 
of all cross-tabulations requested. (Refer to Figure 5 for 
sample of a cross-tabulation.) 

The wiring is very simple, but many wires are needed, 
thus making it profitable to maintain a permanent control 
panel to save the tedium and danger of rewiring for each 
study. A standard alphabetical accounting machine with 
one digit selector and 12 single-position X-distributors is 
required (10 single position Z-distributors may be installed 
at no additional rental charge to replace the five standard 
three-position distributors.) 

Wiring of Control Panel 

The 12 outlet hubs of the digit selector are wired to the 
D inlet hubs of the 12 X-distributors. A card count impulse 
is wired to each of the C inlet hubs of the 12 X-distributors. 
The top row or A and B counters are coupled together to 
form one 40-position counter. The lower row or C and D 
counters are coupled to form a second 40-position counter. 
The controlled side or X hubs of each of the 12 X-distribu- 
tors are wired to the top counter row, allowing three adding 



positions for each. This leaves four positions for the total. 
Each of the 12 counter entries is coupled to the correspond- 
ing entries of the lower counter row. The counter exits are 
then "total transferred" through the four standard class 
selectors and wired to the type bars for the printing of both 
minor and final totals. All of the above wiring is done with 
permanent wires. Now it is only necessary to add one tem- 
porary wire from one of the 80 upper brushes to the digit 
selector entry hub. This wire will be changed as required 
for the column to be distributed. If controls are required, 
two temporary wires are used for each column to be con- 
trolled. 

Advantages of the above method include : 

1. The few temporary wires to be changed to suit the 
particular study can be explained to a person, not 
familiar with the equipment, in a very few words. 
Again, this leads us to our goal of allowing novices in 
the field of punched cards, but experts in statistics, 
to process their own studies economically and safely. 

2. Automatic totals are created for both horizontal and 
vertical breakdowns. This is especially important for 
analysis by chi square. 

3. Increased accuracy through the saving of hand post- 
ing from sorter counters. 

4. In almost all cases where many breakdowns are re- 
quired, the accounting machine method is a tremen- 
dous saving in operator-time. 

Cross Tabulations by Sorter 

In cases where volume is large and breakdowns are few, 
the use of a card counting sorter and hand posting is more 



CROSS TABULATION BY ACCOUNTING MACHINE 





1 


2 


3 


4 


5 


6 


7 


8 


9 





X 


Y 


Totals 


I 


5 


8 


22 


45 


6 6 


1 4 


22 


5 


3 


5 


1 


2 


2 00 


2 


52 


4 6 


2 


32 


1 5 


1 8 


5 


4 


9 


7 








2 13 


4 


175 


5 


6 


8 


9 


4 


1 


2 


50 


4 5 


5 


8 


2 4 7 


7 


12 1 


2 7 6 


3 


5 


1 


100 


8 


70 


42 


12 


2 1 


1 8 


75 9 


8 


5 


8 


9 


4 


3 


5 


7 


9 


12 


42 


5 


2 


4 8 9 


11 


9 


1 


15 


4 


2 


40 


1 


6 


8 


2 








12 4 


95 


12 


8 


4 


100 


5 


^"2X1 


""""^ 


8 


4 





5 


2 


177 


96 


5 


1 4 


12 


1 5 


80 


2 


4 


3 


8 


1 


4 


3 


171 


97 


12 


105 


40 


21 


82 


10 


9 


8 


2 


5 


1 


2 


4 07 


99 


5 


3 








5 


4 


1 


2 





1 





1 


28 


682 


405 


2 00 


195 


32 5 


42 5 


275 


2 7 


152 


72 5 


95 


82 


3 7 6 8 * 



Figure 5 



148 



COMPUTATION 



economical. For this type of study the service bureau sup- 
plies standard forms on which to post cross-tabulations 
from sorter counts. 

Random Sampi^e: Se;IvE:ction 

A good many times in the last two years this bureau has 
been requested to select a random sample or every Nth 
card from a punched card file. This can be accomplished 
easily on the collator, with a card counting device, by wir- 
ing to select every third, fifth, or tenth card as the case 
requires. 

The bureau, not needing the card counting device on the 
collator, maintains a deck of 400 cards in which column 2 
is X-punched in every other card, column 3 is X-punched 
in every third, column 4 in every fourth, column 5 in every 



fifth, etc., to every 40th card being X-punched in column 
40. A consecutive number from 1-400 is also punched to 
guarantee sequence. By placing this deck of cards in the 
secondary feed and the cards from which the sample is to be 
selected in the primary feed, any combination of every 
other, through every 40th card, may be selected by wiring 
from the column required in the secondary brushes to the 
secondary X-pickup. Once prepared, this file may be used 
over and over again and will serve this purpose as well as 
the card counting device. 

It is not the purpose of this paper to bring new discov- 
eries to light, but to illustrate that at least 75 per cent of 
the statistical computations being processed every day in 
the modern institutions of higher learning can be simplified 
and standardized by punched card methods with the co- 
operation of the individual punched card technician. 



Forms of Analysis for Either Nleasurement or 
Enumeration Data Amenable to IS/lachine iS/lethods^ 



A. E. BRAN DT 

Atomic Energy Commission 



A COMMON PROBLEM in statistics is to estimate, 
from a sample, the parameters of a population. If the popu- 
lation is normally or not exceedingly anormally distributed, 
the mean and the standard deviation or the first and second 
moments will adequately describe or specify the population. 
For example, suppose we have a sample of 12 observations 
drawn at random from a given population. The mean and 
the standard deviation, or what is simpler to use, the vari- 
ance of the population can be estimated from this sample. 

A convenient method of securing these estimates is based 
on a matrix-vector product. To illustrate, we shall use the 
random sample of 12 observations previously mentioned. 
Since the sample contains 12 observations, a 12 by 12 ma- 
trix will be required. In order to utilize all of the informa- 
tion in the 12 observations in estimating the population 
mean and variance, an orthogonal matrix must be used. 
Such a matrix can always be written by following a few 
simple rules. First, the terms of the first row of the matrix 
may always be written as a series of plus ones. Second, the 
sum of the terms in each row after the first must be zero. 
Third, the sums of products of corresponding terms in all 
possible pairs of rows, omitting the first, must be zero. It 
will be noted that, under the conditions just stated, if the 
terms of each row of the table are divided by the square 
root of the sum of squares of the terms in that row, the sum 
of the squares of the resulting terms in each row and col- 
umn will be unity. 

The number of such matrices that can be written is very 
large. For example, two such matrices are presented in 
Table I, page 150. 

It is especially important to notice that the first row of 
the matrix provides an estimate of the mean or first mo- 
ment, and that the remaining 11, or 12 minus 1, rows are 
available for estimating the variance or the second moment. 
In general, an « by w matrix of this sort and a single col- 
umn vector of n rows furnish n — 1 comparisons on which 
to base an estimate of variance. I have found this presenta- 
tion very useful for convincing students and research work- 

*This paper was presented by title. 



ers that the use of degrees of freedom in estimating a vari- 
ance is not a matter of choice or what school of statistics 
one follows but of mathematical rigor. They readily recog- 
nize the absurdity of introducing an arithmetic error by 
dividing the sum of w — 1 quantities by n instead of w — 1 
to obtain their mean. This use of « — 1 in the divisor is 
independent of the magnitude of n, so long as n is finite, 
although the difference between the approximate value ob- 
tained by dividing by n and the precise value obtained by 
dividing by w — 1, decreases as n increases. 

The set of values used in the above example was desig- 
nated a random sample. This means that no restrictions 
were imposed on the drawing process or, in other words, 
that the probability that any value in the group or popula- 
tion from which thesq 12 were drawn had exactly the same 
probability in a given drawing of being drawn in that draw- 
ing, as did every other value in the group. The example 
above is of slight interest to workers conducting critical 
experiments, because in the design of such experiments, re- 
strictions upon randomness are deliberately imposed. 

For instance, let^^is suppose that the 12 values used above 
resulted from an experiment on the elimination of weeds 
from a crop such as flax by spraying with chemicals. In 
designing the experiment we knew that two chemicals 
showed definite value as differential sprays, that is, they 
would kill weeds but not injure the crop when sprayed on 
a field. Reports on which was the better were conflicting, 
and considerable doubt remained as to the best period in 
the growing season for spraying. Let us suppose that our 
chief interests center in which is the better spray and 
whether this superiority is constant for the different spray- 
ing periods. Thus, if we spray when the crop is one-quarter 
grown, is one-half grown, and is three-quarters grown, and 
use both chemicals at each spraying, six plots will be re- 
quired. If two replicates are used, 12 plots will be needed. 
If 12 plots of appropriate size and shape are located in a 
field of the crop to be studied, the combination of time and 
chemical to be assigned to each may be determined without 
restriction by some scheme of randomization such as the 
use of random numbers. Experience has shown that the 
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Table I 
















Matrix- Vkctor Product Mkthod for Cai^culating Mean and 


Variance 
























A 




V 

o 




Matrix 
Vector 


Square of 
M-V Prod. 
Divided by 


M-V Prod. 
Divided by 












12 X 12 Matr 


'x 








N 


Q 


Product 


Divisor 


Divisor 


+1 


+1 


+1 


+1 


+ 1 


+1 


+1 


+ 1 


+1 


+1 


+1 


+1 


1 


12 


51 




4.25 


+1 


-1 
































2 


2 


—1 


0.50000 




+1 


+1 


-2 





























3 


6 


—3 


1.50000 




+1 


+1 


+ 1 


-3 


























4 


12 


—6 


3.00000 




+1 


+1 


+1 


+ 1 


-4 























5 


20 


— 10 


5.00000 




+1 


+1 


+1 


+ 1 


+ 1 


-5 




















4 


30 


-5 


0.83333 




+1 


+1 


+1 


+1 


+ 1 


+1 


-6 

















5 


42 


-11 


2.88095 




+1 


+1 


+1 


+ 1 


+ 1 


+1 


+ 1 


-7 














6 


56 


-18 


5.78571 




+1 


+1 


+1 


+ 1 


+ 1 


+1 


+ 1 


+ 1 


-8 











7 


72 


-26 


9.38889 




+1 


+1 


+1 


+ 1 


+ 1 


+1 


+ 1 


+ 1 


+ 1 


-9 








6 


90 


-17 


3.21111 




+1 


+1 


+1 


+ 1 


+ 1 


+1 


+ 1 


+ 1 


+ 1 


+ 1 


-10 





5 


110 


-7 


0.44545 




+1 


+1 


+1 


+1 


+ 1 


+1 


+ 1 


+ 1 


+ 1 


+ 1 


+1 


-11 


3 


132 


15 


1.70455 




34.24999 




Variance 


__ 




sum of quotients 






34.24999 


= 3.1136 














one less 


than 


number of 


rows 


in vector 


11 












+1 


+ 1 


+ 1 


+ 1 


+ 1 


+1 


4-1 


+1 


+1 


+1 


B 

+1 


+ 1 


1 


12 


51 




4.25 


+1 


— 1 
































2 


2 


—1 


0.5 










+1 


-1 


























3 


2 


—1 


0.5 
















+ 1 


-1 




















4 


2 


+ 1 


0.5 






















+1 


—1 














5 


2 


—1 


0.5 




























+1 


-1 








4 


2 


+ 1 


0.5 


































+1 


-1 


5 


2 


+2 


2.0 




+1 


+1 


-1 


-1 


























6 


4 


—4 


4.0 
















+1 


+1 


-1 


—1 














7 


4 


—2 


1.0 




























+1 


+1 


-1 


-1 


6 


4 


+5 


6.25 




+1 


+1 


+1 


+1 














—1 


-1 


-1 


-1 


5 


8 


-11 


15.125 




+1 


+1 


+1 


+1 


-2 


-2 


—2 


-2 


+1 


+1 


+1 


+1 


3 


24 


-9 


3.375 




34.250 




Variance 


34.Zb 
11 


3.1136 























material, time, and money required for this experiment can 
generally be more efficiently used if certain restrictions are 
placed on the assignment of treatment to plots. 

A restriction on randomness might be imposed by divid- 
ing the 12 plots into two compact groups or blocks of six 
plots each. In general the plots should be comparatively 
long and narrow, and the blocks should be square or as 
nearly so as practical. The six combinations of two chemi- 
cals and three spraying times can now be assigned by a 
scheme of randomization to the six plots in each block, but 
the two assignments must be separate and independent. 
Since we are chiefly interested in the comparison between 
the two chemicals, we should further restrict the method of 
assignment so that the two chemicals will always appear 
next to each other. This is done by assigning the three 
times of spraying at random to three main plots, of two 
subplots each, in the two blocks, and then assigning the 
two chemicals to the two subplots in each main plot inde- 



pendently by some scheme of randomization, such as flipping 
a coin. 

The effect of these restrictions on randomness is to re- 
duce the number of matrices that can be used, compared to 
the vast number of matrices possible when no restrictions 
are imposed. One of this limited number of possible mat- 
rices is presented in Table II. In this table the observation 
vector, consisting of values representing the efficacy of 
treatment on each plot, is placed at the top so that its rows 
correspond to appropriate columns of the matrix ; further, 
the design of the experiment — but not the field layout — and 
hence the analysis of the data is specified. Terms of -|- 1 and 
— 1 are represented by -|- and — alone, and those of zero 
by blanks. 

It is at once evident from the examples presented that in 
this form of analysis of research data, the number of com- 
puting steps is large and the number of data small. I under- 
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TABI.E II 
















Matrix- Vector Product Method of Doing Anai^ysis oe 


Variance Cai^culations 




Block 
Time 








I 










II 




te 


<-5 


Matrix- 


Square of 
M-VProd. 


M-VProd. 


early 


middle 


late 


early 


middle 


la 


Chemical 


a 


b 


a 


b 


a 


b 


a 


b 


a 


b 


a 


b 


:s 


Vector 


Divided by 


Divided by 


Observation Vector 


1 


3 


4 


2 


5 


4 


6 


5 


7 


6 


5 


3 


Q 


Product 


Divisor 


Divisor 


comparisons 


































first moment 


































mean 


+ 


+ 


+ 


+ 


+ 


-f 


+ 


+ 


+ 


+ 


+ 


+ 


12 


51 




4.25 


second moment 


































main plots 


































block B 


— 


— 


— 


— 


~ 


— 


+ 


+ 


+ 


+ 


+ 


+ 


12 


13 


14.0833 




time 


































linear T\ 


+ 


+ 






— 


— 


+ 


+ 






— 


— 


8 


-2 


0.5000 




quadratic T2 


+ 


+ 


-2 


-2 


+ 


+ 


+ 


+ 


-2 


-2 


+ 


+ 


24 


-6 


1.5000 




error (a) 


































BXTi 


— 


— 






+ 


-f 


+ 


+ 






— 


— 


8 


8 


8.0000 




BXT2 


— 


— 


+2 


+2 




— 


+ 


+ 


-2 


-2 


+ 


+ 


24 


-8 


2.6667 




sub-plots 


































chemical 


































C 


— 


+ 


— 


+ 


— 


+ 


— 


+ 


— 


+ 


— 


+ 


12 


-5 


2.0833 




interaction 


































CXTi 


— 


+ 






+ 


— 


— 


+ 






+ 


— 


8 


4 


2.0000 




CXT2 


— 


+ 


+2 


-2 




+ 


— 


+ 


+2 


-2 




+ 


24 


4 


0.6667 




error (b) 


































BXC 


+ 


— 


+ 


— 


+ 


— 


— 


+ 


— 


+ 


— 


+ 


12 


+3 


0.7500 




BXCXTi 


+ 


— 








+ 


— 


+ 






+ 




8 


-2 


0.5000 




BXCXT2 


+ 




-2 


+2 


+ 






+ 


+2 


-2 




+ 


24 


-6 


1.5000 




34.25 



stand that the card-programmed electronic calculator has 
been built to fit this situation. The number of observations, 
which directly reflects the size of an experiment, that can 
be handled by the CPC is determined by the capacity of 
the internal storage. 

Frequently, the number of observations resulting from 
an experiment is much larger than the internal storage of 
the CPC can accommodate, but a less detailed analysis 



than that yielded by the above method will suffice. In such 
cases, whether they be counts or measurements, the data 
can be reduced by machine methods by employing sorting 
and tabulating techniques more fully. The same set of 12 
observations will be used to illustrate the method. Only 
a few of the necessary machine details will be given in 
Table III (a, b, c). The number of runs or subtables re- 
quired for this scheme is equal to the number of major divi- 



TABI.E III 










Step Method oe Doing Anai^ysis oe Variance Cai^cuIvATions 


(part a) 












Chemical 


Time 


Block 


Total 


1 
3 


4 










4 












2 
5 


6 










4 

6 


9 




19 






5 
7 


11 










6 

5 


13 










3 


8 




32 


51 


number of entries n 


12 


6 




2 


1 


number of values per entry k 


1 


2 




6 


12 


number of degrees of freedom d/f 


12-6=6 


6-2=4 


2-1= 


=1 




sum of squares of entries 


251 


487 


1385 




2601 


sum of squares of entries over k 


251.0 


243.5000 


230.8333 


216.75 


correction terms 
difference 


243.5 


230.8333 


216.7500 




7.5 


12.6667 


14.0833 


C + C XT -{-C X 


B -\-CXTXB= 7.5 
T-\-TXB = 12.6667 










B = 14.0833 
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TABI.F 


III [continued] 










(part b) 












Time 
1 


Block 


Chemical 


Total 




4 
5 
6 

7 
5 
3 




10 
18 


28 






2 
4 
5 
6 
3 




9 

14 


23 


51 


number of entries n 
number of values per entry k 
number of degrees of freedom d 
sum of squares of entries 
sum of squares of entries over k 
correction terms 

difference 

T -\-T X B + T 


12 
1 
// 12-4=8 
251 

251.000 
233.6667 


4 
3 

4-2=2 

701 

233.6667 

218.8333 


2 
6 
2-1=1 
1313 

218.8333 
216.7500 

2.0833 
3333 
8334 
0833 


1 
12 

2601 
216.75 


17.3333 14.8334 
XC + TX5XC = 17. 
B-LBXC = 14. 
C= 2. 




(part c) 












Block 


Chemical 


Time 


Total 




1 
6 
3 
5 
4 
7 
2 
6 
5 
5 
4 
3 




7 

8 

11 

8 


15 
19 








10 

7 


17 


51 


number of entries n 
number of values per entry k 
number of degrees of freedom d 
sum of squares of entries 
sum of squares of entries over k 
correction terms 

difference 

5 + i? X C + 5 


12 
1 
// 12-6=6 
251 
251.0 
223.5 


6-3= 
447 
223.5C 
218.75 


6 

2 

=3 

27. 
4. 
2. 


3 
4 

3-1=2 

875 

218.75 

216.75 


1 
12 

2601 
216.75 


27.5 4.75 

X T-^BXC-\-T = 

C + CX T = 


2.00 

5 

75 

00 


C = sum of squares attributable to differences between 

chemicals 
T =sum of squares attributable to effects of time 
B = sum of squares attributable to block differences 
C X T = sum of squares attributable to interaction between 

chemicals and times 
T X B = sum of squares attributable to interaction between 

times and blocks 
C X T X 8^== sum of squares attributable to interaction between 

chemicals, times, and blocks 





sions of the variance to be made which is three in this case. 
All but the last column in each table will be headed by a 
variable or major division, and the order of sorting will be 



determined by reading these headings from left to right. 
After the first subtable has been made, only one sort is re- 
quired for each of the others. In each subtable the first 
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Tablu IV 








Summary of Analysis 




Source of Variability 








DIP 


5Mm o/ 
Squares 


major plots 
block B 








1 


14.0833 


time T 








2 


2.0000 


error (a) 5 X T 








2 


10.6667 


minor plots 
chemical 






C 


1 


2.0833 


interaction 




C X 


T 


2 


2.6667 


error (b) 5 X C + 
Total 


B 


X C X 


T 


3 
11 


2.7500 


34.2500 



column is formed by listing all observations, the second 
consists of subtotals secured by minor controlling on the 
variable heading the column, the third consists of subtotals 
secured by major controlling on the variable at its head, 
and the fourth is the grand total or sum of all observations. 

From a combination of the three equations at the bottom 
of each subtable, the values of the components of variability 
defined above can be determined. To clarify the notation, it 
might be well to point out that C xT X B = T X B X C 
= B X C X T, etc. The solution of this set of equations 
may be summarized as shown in Table IV. 

The sum of squares for the divisions of the total sum of 
squares in Table IV can also be secured from Table II. 



The details on how to adapt machines to this step method 
will depend, of course, on the machines available. With only 
a sorter and accounting machine, the procedure would fol- 
low that indicated in IIIa, IIIb, IIIc, with considerable use 
of a table model calculator being required. With higher 
types of machines at hand, the alert operator will readily 
adapt them to the problem. 

DISCUSSION 

Chairman Hurd: One idea, that has appealed to me par- 
ticularly, was the first idea described. One has this matrix, 
Table I. In order to compute the sums of squares, one needs 
to multiply the matrix by the vector, essentially. So if one 
had a number of experiments which were like this in form 
— a number of factors — one would punch a deck of program 
cards in which each card has an operation which is either 
add or subtract. That is clearly all that is required; and, 
given an experiment, you punch one card for each observa- 
tion, drop these in the calculator, followed by this program 
deck which performs the matrix-by-vector multiplication, 
and you have sensibly analyzed it. 

Dr. Brandt: If you wish to do a multiple covariance cor- 
relation or multiple regression with this method, all you 
have to use is a multiple-column vector, each column of the 
vector representing one of those variables. As it was here, 
I had merely the one variable, and so I used a one-column 
vector. But I can have a vector with a good many columns, 
if I desire. 



Remarks on the IBM Relay Calculator 



* 



MARK LOTKI N 

Ballistic Ke search Laboratories, Aberdeen Proving Ground 



ASIDE from its large-scale digital computers, such as the 
ENIAC and the Bell Relay Computer, the Computing Lab- 
oratory of the Ballistic Research Laboratories, Aberdeen 
Proving Ground, has at its disposal some special high-speed 
computing devices built by IBM for the Ordnance Depart- 
ment, the IBM Relay Calculators. Two of these machines, 
identical in every respect, were installed in December, 1944, 
and they have been in continuous operation ever since, ex- 
cept for short periods of time when the incorporation of 
certain improvements necessitated their shutdown. 

While there are presently in existence three other IBM 
Relay Calculators — one at the Naval Proving Ground at 
Dahlgren, Virginia, and two at the Watson Scientific Com- 
puting Laboratory, Columbia University — the Aberdeen 
relay calculators have acquired additional interest owing to 
certain modifications that have been made lately on these 
machines. I am referring here to the hooking up of the twin 
machines, thus effectively transforming them into a single 
device of greatly increased computing power. 

To arrive at a better understanding of the "coupled" cal- 
culator, as it is presently constituted, it seems best to ac- 
quaint the reader first with the principal features of these 
sequence-controlled, digital relay calculators, and then dis- 
cuss some of the modifications resulting from the hookup. 
For more detailed descriptions of the relay calculators the 
reader is advised to study the recently published report of 
J. Lynch and C. E. Johnson, "Programming Principles for 
the IBM Relay Calculators," Ballistic Research Labora- 
tories Report No. yoj, October, 1949. 

The Relay Calculator 

Data, to be introduced into the storage or computing 
registers of the machine, must be punched in decimal form 
on standard IBM cards; each number may have up to 12 
digits with a plus or minus sign. Two separate card feeding 
units, the reproduce feed and the punch feed, each operating 
normally at the rate of 100 cards a minute, are associated 
with five reading stations and one punching station. Four 
of these reading stations permit as many as four cards to 
be read simultaneously, while results are punched on a fifth 
card. The fifth reading station provides a means of reading 

*This paper was presented by title. 



from the cards certain punches which may be used to con- 
trol the sequencing of operations to be performed. This pos- 
sibility of programming by means of control cards, in addi- 
tion to the usual wiring of control panels, will be discussed 
in greater detail later on. 

A comparing unit, pluggable to reading brushes as well 
as counters, permits the comparison of two sets of data each 
having as many as 80 columns. Thus, data punched in two 
decks may be compared, and the machine may be instructed 
to stop when such a comparison reveals a, disagreement. 

There is, further, available in the machine a collating 
unit, of importance when it is desired to compare six-digit 
or 12-digit numbers punched in cards or stored in counters. 
Depending on the result of this comparison, the machine 
may then be instructed, by means of class selectors, to start 
certain operations such as card feeding, transfers, etc. 

The timing of all operations performed within the ma- 
chine is regulated by a shaft in the card feeding unit rotat- 
ing at the rate of 100 rpm. During one revolution of this 
shaft, a "machine cycle," exactly one card may be read or 
punched. During the same revolution a cam timing circuit 
produces 48 impulses that may be used to activate relays. 
These sequence impulses, emitted at regular time intervals 
of 1/80 of a second, form the basic sequence of "points" 
that governs the timing of all machine operations. A point 
represents the approximate length of time required for the 
proper operation of the relays. 

While the timing unit, just mentioned, produces the 48 
relay impulses, a second unit emits 12 digit impulses ; these 
are of importance for the reading, comparing and punching 
of cards, and in the coupled calculator, also, for the control 
of operations. For the proper time sequencing of operations 
it is important to keep in mind the time of occurrence of 
these digit pulses within the card cycle. This basic relation- 
ship is shown in Figure 1. 



Digit 



■f^ 



Sequence 



f^ 



. rTi. .n 



-Pn 



rj;^ 



18 



24 



30 



36 



42 



Figure 1. Suquknce Pulses and Digit Pui^sss 
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While the basic mode of operation is of the single-cycle 
type, it is possible, in cases where the normal sequence of 
48 points is insufficient to permit the performance of series 
of operations required in the solution of a problem, to run 
the machines on the double-cycle basis. Only 50 cards then 
are read per minute, but the machine carries out approxi- 
mately twice as many operations as it does during the single 
cycle setup. The storing and computing of numbers in the 
machine is effected entirely by means of approximately 
2,500 electro-mechanical relays arranged in 31 counters, of 
which five can have 12-digit signed numbers, and the other 
26 can accommodate six-digit signed numbers. Numbers 
are stored in the counters by a "pentad" system ; in this 
type of arrangement each decimal digit is represented by a 
certain position in a "column" consisting of live relays, as 
shown in Figure 2. There the configurations a, b, . . . , f 
represent, respectively, the decimal digits 0, 1, 2, 5, 6, 9. 
Thus, a number of six digits requires the services of 30 
relays, aside from the sign relay. 
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Figure 2. The "Pkntad" System 



In addition to the 31 counters there are 24 dial switches, 
each capable of storing one digit. The machines, therefore, 
have an internal storage capacity of 240 digits each, or 
twice that number when operated in tandem, as described 
below. 

The different counters can perform some, but not all, of 
a variety of functions. Such functions have to do with the 
receiving of data from the switches, the cards, or other 
counters ; transmitting data to other counters or the punch 
magnets ; cooperating with other counters in the formation 
of new quantities such as sum, product, etc. ; shifting num- 
bers to the right or left ; rounding and comparing of results. 
All these operations are carried out at relatively high speed. 
Thus, the time required for the addition or subtraction of 
two signed numbers, having as many as 12 digits, an opera- 
tion which requires three counters, is two sequence points. 
For multiplication of two numbers of six digits each, result- 
ing in a 12-digit product, 16 points are needed ; this ties up 
10 counters. The division of a 12-digit numerator by a six- 
digit denominator, resulting in a quotient having as many 
as 12 digits, takes, on the average, 0.2 seconds per quotient 
digit. This operation engages six counters. It takes about 



the same amount of time, but only four counters, to obtain 
one of the six digits that form the square root of a 12-digit 
number. 

Now, a few words about the two control panels of the 
relay calculator. y\s mentioned previously, all operations 
carried out within the machine are synchronized with a tim- 
ing unit which emits 48 sequence impulses during one revo- 
lution of a rotating shaft, or 96 impulses when double-cycle 
operation is called for. These impulses, coming in succes- 
sion, are routed to 96 sequencing relays, each of which is 
associated with a number of outlets located on the right- 
hand section of a two-section control panel. Sequence im- 
pulses through these outlets, thus, can be made to control 
the sequencing of operations. The left half of the two- 
section control panel, on the other hand, serves to decide 
the type of operation to be performed on each sequence 
point. 

The general purpose of the other control panel, consisting 
of three sections, is to give instructions pertaining to the 
reading, comparing, and punching of data, and to assign the 
channels through which the data can be routed. 

Coupling of the Relay Calculators 

While, in general, very fast and efficient, the machines, 
when operated singly, suffer from two shortcomings that 
quickly become apparent when problems of a more complex 
nature are to l)e solved : they are rather limited in both 
storage and programming facilities. 

Now these limitations have been greatly mitigated. C. E. 
Johnson, in cooperation with C. B. Smith of IBM, has 
worked out a method that utilizes both machines simul- 
taneously in a manner designed to minimize the deficiencies 
cited above. Coincidentally, work is progressing on an all- 
purpose control panel to be used when the machines are 
operated in tandem. The two machines are connected by the 
installation of a single connector cable between the main 
units of each machine. The changes in the internal wiring 
of both machines necessary for this type of coupling were, 
however, made in such a way that by simply disconnecting 
the cable, both machines may be used independently as done 
prior to the connection. When working in tandem, one of 
the calculators acts as a storage organ only, while the other 
one takes over the computing functions. Merely by a switch 
of the ends of the connector cable, the roles of storage unit 
and computing unit also become interchanged. Obviously, 
this type of arrangement has much to recommend it, since 
it may permit continuation of a problem without interrup- 
tion in the event certain types of machine failure occur. 

In tandem operation, the machine that supplies the se- 
quence impulses is the computing organ, while the machine 
receiving these impulses becomes the storage organ. 

Operational control may be exercised entirely by punches 
on cards, through the use of class selectors, which are emi- 



156 



COMPUTATION 



nently suited for variable sequencing of operations. Each 
machine has 25 such selectors, of which six have three posi- 
tions, and 19 have six positions, with four of the latter cate- 
gory being of limited applicability. Twenty-one of these 
selectors are presently being used when problems are solved 
on the coupled calculator. A further enlargement of pro- 
gramming facilities has been achieved by splitting up each 
six-position class selector into two class selectors of three 
positions, thus obtaining a total of 72 three-position selec- 
tors, a number which seems to be quite adequate for most 
of the jobs usually considered for the calculator. 

Also, aside from code punches located in pre-assigned 
columns, the control cards are punched to govern the oper- 
ation of the feed and punch units of both individual units ; 
they may, moreover, contain constants that are needed 
in the course of the computation. Any of the symbols 
Y, X, 0, 1, . . . , 9, in any of the 80 columns of a standard IBM 
card, may be used as code in the card control. Thus, a y in 
column 3 may run the punch feed of the storage unit, an x 
in column 59 may instruct counter 10 to transfer out, a in 
column 64 may initiate the performance of a certain shift, 
etc. The choice of control punches is quite arbitrary and 
may be made to suit the requirements of each particular 
problem, in conformity with the wiring of the control panel. 

The sensing of the code punches by the reading contacts 
of the calculator initiates a signal which, properly routed 
through the class selectors, starts a certain routine pre- 
viously wired on the control panel. A deck of control cards, 
then, placed in one of the four feeds of the computer, will 
govern the sequences of operations into which the problem 
has been broken previously. 

In a typical setup the calculator will operate as follows : 
first, input data are punched on cards and placed into the 
reproduce {Rg) and punch {Pg) feeds of the storage unit. 
Reproduce {Re) and punch {Pc) feeds of the computer, on 
the other hand, contain the deck of control cards and blank 
cards, respectively. The reading of the first control card 
causes Rg to feed a card, thus introducing initial data into 
the machine. The following control cards cause the opera- 
tions to be carried out in the desired sequential manner. 
Ultimately, the last control card instructs Pc to punch the 
final result. Now the sequence of the operations is started 
again by the reading of the first control card, which causes 
a new set of initial data to be introduced through Rg, etc. 
To preserve the continuity of operations, it has been found 
advisable to produce not one, but a sufficient number of 
such control decks, thus avoiding a delay, otherwise en- 
countered, because of the necessary handling of the control 
cards. 

Examples of Problems 

The coupled calculator has been in operation now for 
over a year, and the results have been extremely satisfac- 
tory. Many of the problems, whose complexity made their 



sohition on the relay calculators impractical, are now han- 
dled speedily and efficiently. Thus, the calculation of the 
instantaneous position in space of a moving body by means 
of theodolite observations has been achieved in slightly over 
two minutes. 

In another problem it was necessary to solve a system of 
32 linear equations arising from a triangulation problem. 
To give an idea of the variety of types of computations now 
possible with the relay calculators, perhaps, it may be 
worthwhile to present a short account of some of the phases 
into which the solution of a problem of this latter type may 
be divided conveniently. 

After carrying out the measurements previously decided 
upon, and inserting the conditions inherent in the geometry 
of the problem, there will result certain conditional relation- 
ships represented by p linear equations in n unknowns v^ : 



n 

fi = ^aifct^fc + n = 0, i = 1, 2, . 



,P- 



(1) 



The solution of ( 1 ) may be achieved by the minimization 
of '^vi, subject to constraints imposed by (1). Introducing 
p Lagrange multipliers, Ci, we then must make 

F^^ {vll2) -^Ci/i = min. (2) 

fc i 

Now the equations dP/dvy, = necessary for this minimi- 
zation lead to 



Vk 



V 



O'jkCj. 



(3) 



j=i 



These expressions, in turn, after being introduced into ( 1 ) , 
lead to the normal equations 

\AijCj + ry = 0, y = 1, 2, . . . , p, (4) 



with 



Aii 






O'ik^^jlc- 



(5) 



Once equations (4) are solved, then the v^, as computed by 
(3), will satisfy (1). 

In the carrying out of actual calculations, the following 
remark is of importance for the checking of the normal co- 
efficients : if we put 

Sic = /_^%fc, ^ = 1, 2, . . . , w, (6) 

and 

i 
then, obviously, 

Si = y O'iTcSk- 



1,2,. ..,/>, 



(7) 



The first phase of the problem, then, deals with the calcu- 
lations of the normal matrix, a job which may be handled 
on the relay calculators in straightforward fashion. The 
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second phase, i.e., the solution of equations (4), however, 
may be done in a variety of ways. We have found Jordan's 
method, which is a modification of Gauss' method of elimi- 
nation, most suitable for treatment on the machines. Applied 
to our matrix (Aij,ri), i.e., (Ai/) augmented by the column 
Ti, this method proceeds as follows : 

Step I. Find the Ai^ of largest absolute value. Without 
loss of generality this coefficient may be labeled A^.^. Then 
by using the factors 

it is permissible to transform the matrix {Aij,ri) into the 
equivalent matrix (A^p, r'-P ) by means of the relationships 



(8) 



Ail> = Aii - m['^A,j, 

r^^') = n — w^J^Vi, i = 2,3, . . . , p. 
Accordingly, 

A^l^ =0, while ^il> =A,,. 
Note also that if Aji = Atj then Alp = A[p for i, ; > 1. 

Step 2. Find the ^^^^ of the largest absolute value, for 
i = 2, 3, . , . , p. This coefficient may be labeled ^22'^ with- 
out loss of generality. Form the factors 

w^,^> = AIP/A<^1 (9) 

and put A^p = AiP - m[^'A^}\ 

^(2) _ f. 



(2) _ r<v — mf'r'i\ i = I, 3, A,.,., p. 



Then 



A\r = 0, while .4^1> = Ail 



Also, A[l' = All^ = A,^, but AlP = for i ^ 1, 

since A{1^ =0 for these i. 

Thus, the transformed matrix has the form 

lA<iV A?,' .... A\-j r\ 



J(2) /l(2) 

-^22 -^23 

A(2) 



J<2) 

/i(2) 
^2p 

■^3p 



\o A^-p .... A^p 

Again, if the original matrix Aij is symmetric, then A^p = 
Af for i, i > 2. 

Generally, let us assume that the first e columns have 
been so transformed. Among the ^(e+i, i = e -\- 1, e -\- 2, 



. . . , p, let ^e+'i.e+i be the one of largest absolute value. Then 
the relations 






(10) 



will carry the diagonalization process one step further. 
After p such steps the matrix {Aij, ri) will look like this : 

lAiV r'*^ 







AW 











r.V 



(p) 



(11) 



^pp 



fp 





The desired solutions are, therefore, 

cj = - rf^//A^S ■ 
For purposes of checking calculations it is useful to carry 
along at each step the sums of all coefficients in each row. 
If, namely, one computes at the start 



Ti = -{y2iAii + rA 



(12) 

and, during step 1, transforms the Ti together with the r</' : 
r/i> = Ti-m[^'T^, (13) 

then, obviously, 

^^n) + ^(1) ^ 7;a) ^0. (14) 

/ 

A similar relationship holds for each step. 
Once the first solution c^^^ ^ {^''P , c^^\ - • • , Cp^^) has 
been obtained, it takes considerably less time to get an im- 
proved solution c ~ c^^' -\- d. If, namely, equations (4) are 
written in matrix form : 

Ac + r = 0, 
and the residuals due to c^^^ are t ^ {t-^, t.^, . . . , tp) : 

Ac^^^ +r = t, 
then Ad-\-t = 0, 

and since A has been diagonalized previously, it is necessary 
only to transform the column t. 

The relay calculators, completing a whole step with each 
machine run, took about 20 hours to solve the problem with 
the desired degree of accuracy. From the experience gained 
thus far, it may be stated that the coupling of the IBM 
Relay Calculators has resulted in a decidedly superior com- 
puting machine. 



An Improved Punched Card Nlethod for 
Crystal Structure Factor Calculations^ 

MANDALAY D. GREMS 

General Electric Company 



THOSE of you who have worked with calculations deal- 
ing with the structure of complex crystals, are reminded, 
probably, of the long monotonous operations involved. For 
this reason, a few persons here and there have attempted to 
find methods for simplifying the tremendous amount of 
hand calculations. Shaffer, Schomaker and Pauling, at the 
California Institute of Technology, were the first to report 
a method using the IBM equipment for this purpose. How- 
ever, at our own Research Laboratory at the General Elec- 
tric Company in Schenectady, there is a group of scientists 
who have spent considerable time and effort on this work, 
both analytically and theoretically. 

After a few discussions of their problem, it seemed more 
efficient and better suited to the IBM equipment to begin 
with the general expression, 
y 

Phki = 2^/iCos2ir {hXj-\-kyj-\-lZj) 



j=i 



rather than to use a specific and modified expression for 
each type of structure factor calculation. 

This expression doesn't look difficult until you consider 
that it involves many combinations of the reflections h, k, I 
with the trial parameters x, y, s to find the best sets of 
X, y, s. 

At the beginning, three separate decks of cards are key 
punched : 

1. Table cards. For sin lircn. and cos 2-n-a, where a ranges 
from 0.001 to 1.000, in intervals of 0.001. This pack 
is used for all crystal structure calculations. 

2. Reflection cards. One card for each reflection h, k, I. 
This card also contains the scattering factor for each 
kind of atom, the temperature factor, and the absorp- 
tion factor (if known) for that particular reflection. 
These reflection cards are used for all trials for a 
specific crystal structure factor. 

*This method was presented at the American Society for X-ray and 
Electron Diffraction in Columbus, Ohio, on December 16, 1948. It 
also appeared in the December issue of Acta Crystallographica, 
Vol. 2, Part 6. 



II. 
III. 



+ » ^/y sin 27r ( hXj-{-kyj-\- kj) ly 



3. Parameter cards. One card for each set of trial param- 
eters X, y, z. The number of cards depends upon the 
unit of structure. These cards are used for a specific 
calculation. 

First, reproduce the set of reflection cards as many times 
as there are sets of parameter cards, gang punching a set of 
X, y, z values on each reflection deck. If there are 400 reflec- 
tions and eight sets of parameters, then there are 3,200 de- 
tail cards each containing an h, k, I, x, y, and z. 

There are four main machine operations in the solution 
of this problem. The two important steps, or the two con- 
tributing the most to a more compact and general proce- 
dure, are steps I and III. 

I. Forming the quantities a, = (hxj -\- kyj -\- Izj) 
Obtaining the cosines and sines of ay 
Multiplying the trigonometric functions by the 
scattering factors, fj 
Summing the previous products 

Step I indicates the formation of the quantities Oy = 
(hxj-j~kyj-\-lZj) . Using the IBM Type 602 Calculating 
Punch with the above detail cards, it is possible to find ay, 
bj, Cj, and dj at the same time — ^that is, with only one 
passage of the cards through the machine, where 

aj = hxj -f kyj-\- Izj , 
bj = hxj -f- ky) — Izj , 
Cj = hxj — kyj -\- Izj , 
dj = — hxj -f kyj + Izj . 

As the next step involves looking up the sine or cosine of 
the quantities a, b, c, and d, it is sufficient to carry only the 
decimal places in the product and sums. Therefore, multiply 
h by X, and carry the three decimal places to the four sum- 
mary counters, adding the product in counters 14, 15, and 
16 and subtracting the product in counter 13. Then multi- 
ply k by y, add the three decimal places of the product 
into counters 13, 15, and 16, and subtract it into counter 14. 
In the same manner, multiply / by z, add the three deci- 
mal places of the product into counters 13, 14, 16, and sub- 
tract it into counter 15. To eliminate the possibility of cal- 
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culating a negative value, add 1 .000 in each of the counters 
13, 14, 15, and 16 on the reading cycle. Now it is unneces- 
sary to include negative a's in the sine and cosine table. 
When these four sums are punched, each card contains a 
positive number for a, b, c, and d. 

For certain symmetry elements the structure factor will 
contain any or all of the terms b, c, d, as well as a ; so there 
is a considerable reduction in the number of atomic param- 
eters necessary when all can be found at one time. 

This, also, makes the procedure general for most struc- 
tures. The effect of symmetry is illustrated by the sDaee 

group Pmmm , 

atoms 2X X y z, X y'z, x'y z,'x y z 
xyz, xyz,xyz,xyz, 

N/8 

Plikl = 8/ f j (cos lira j -\- cos lirbj -\- COS ZttCj -f- COS 2Wy). 



i=l 



As you notice, the structure factor can be written in 
terms of a, b, c, d. Therefore, the parameters of only 1/8 of 
the atoms in the unit cell need be considered. 

Now look at another space group Pnnm, for example : 
For (/i + ^ + /) = 2w , 

N/S 

P^y^l = 8y /;(C0S liraj -f- cos 2trbj -\- cos 2TrCj -\- cos 2-7rdj). 

i=x 

Vox {h -^ k -\- I) =2n+ I, 

N/8 

pj^y.1 = 8^ /i(cos 2iraj -f cos 2Trbj — cos 2-7rCy — cos 2Trd}). 
i=l 
For this space group, there are two diilferent expressions 
for Pmi, depending upon whether {h-{-k-\-l) is odd or even. 
They both contain a, b, c, and d, but the algebraic combina- 
tions of the cosines differ. This does not change our general 
procedure, however, and it is a simple matter for the ma- 
chines to separate the two groups at the proper time. 

At step II, use the previously prepared sine or cosine 
table deck; sort, collate, and intersperse gang punch the 
table values on the detail cards for the four factors a, b, c, d 
where 

cos 2traj — Aj , 

cos 2irbj = Bj , 

cos 27rCj = Cj , 

cos 2'7rdj = Dj . 

Each detail card should now contain the following: 
h,k,l,x,y,z,a,b,c,d,A.,B,C,D as well as a code for the type and 
number of the atom. 

Until step III there was no need for a particular ar- 
rangement of the cards. At this point, the cards must be 
sorted on the column indicating the kind of atom. The dif- 
ferent kinds are kept separate, and these packs are sorted 
according to reflection h, k, I. 

Using the accounting machine with the summary punch 
attached, punch on blank cards the {A, B, C, D) for each 



reflection. Repeat this operation for each kind of atom sepa- 
rately, where 

/ ^COS 2-7raj = /^^i = Amy , 

and 

y ^cos 27ra2 = / ^A^ = Am2, etc. 

Therefore, each summary card for atom ( 1 ) now contains 
the code for kind of atom, the reflection, and (A,B,C,D) ml; 
summary card for atom (2) contains (A, B, C,D) m2 sub- 
stituted for {A, B, C, D) ml. 

At this time, it is necessary to refer to the expression for 
the particular structure being studied, in order to determine 
how to combine {A, B, C, D) ml or w2. 

Referring to space group Pmmm, {A-\-B-\-C-\-D) m^ 
or m^ is necessary. 

Referring to space group Pnnm, {A-\-B-\-C-]rD) w^ 
or m^ for 2« ; and 

{A-\-B) — (C+£>) m^ or m^ for 2w + 1. 
This is only a minor change on the IBM type 602 control 
panel to perform either operation. Another variation for a 
complex group can be done simply and easily at this time, 
if both P^u and P%^i are required. 

The sums of the y^, 5, C, and -D's can be calculated for 
both at the same time. Only one set of reflection cards {hkl) 
are required until the final stages of the work. 

After the A, B, C, and D's are combined properly, the 

sum is multiplied by the proper scattering factor, jmi or jm^- 

jm^{A-\-B-\-C-^D) Wi = 7?Wi,etc. 

The final step consists of simply adding these products 

together for the proper reflection and multiplying by a 

factor for that reflection, 

Thu{Ri^i + ^Wg) = Phkh 
It is usually of interest to note the contributions of 
each kind of atom to the final result ; so it is advisable to 
list the factors Rm^ and Rm^, as well as Pnu, on the record 
sheets. 

DISCUSSION 

Mr. Smith: How long did it take you to calculate, say, 
for the order of 600 reflexes for your space group Pnnn or 
Pmmm , OT that Order between 100 and 600? 

Miss Grems: It first took me twice as long as it did later, 
because I checked, and after I had done quite a number of 
these I found there was no point in checking the calculation 
of the quantities hx -\- ky -\- Iz or a, b, c, and d, because if 
there was an error it wouldn't make much difference. I 
would say roughly about three and a half days — perhaps 
four or four and a half. It added about an extra day to carry 
on the checking, although a good part of the checking could 
go on at the same time. I always did check the last part of 
calculations after I found the A's. Of course, it was a simple 
check. 
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Mr. Smith : That was for roughly about 500 reflexes ? 

Miss Grems: That is right; and breaking it down to 
about eight y, x, s's. 

Mr. Smith: That would be roughly about a fifth of the 
time it would take you with a hand calculator, maybe less ? 

Miss Grems: For the particular case about which I was 
talking, we found for both the P^u plus and Phu minus, it 
took only a half-hour longer to get the Pnu minus. 

Chairm,an Hurd: Is the method which you have used, 
Mr. Smith, roughly analogous to this ? 

Mr. Smith: Unfortunately, no. I have been using a 
method somewhat similar to the one they use at California 
Tech., which differs somewhat from this ; and, unfortun- 
ately, most of it has been done on the hand calculator. Also, 
unfortunately, the last case, instead of having, say, eight 



terms, had twenty terms in the general space group. It was 
a little more involved than that, but I was able, by using 
some Fourier transforms, to eliminate the necessity of cal- 
culating those two longer terms. 

Mr. Thom.pson: Regarding the layout cards for master 
cards, which most of us use, our local IBM man made a 
very good suggestion of which some of us may not have 
thought. He suggested that we punch every hole in the card. 
When you want to read a detailed card, you put the layout 
card right over the detail card as a mask, and this makes it 
very quick to read. A couple of warnings, however : When 
you do this, don't punch every hole at once. If you punch 
them all simultaneously, two things might happen. The 
punches may stick in the dies or, as a matter of fact may 
come out of the left-hand side. It is advisable to send them 
through about eight times. 
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THE hypergeometric function Pia,h;c;s) is usually de- 
fined, for purposes of calculation, by the hypergeometric 
series : 



F{a,h -yC^z) 



1 I ^'^ 



a(a+l) h{h-\-\) 



. a(a-H)(a+2)&(&+l)(&+2) 

-\ 1 ^ o — 7 — . i N / — r-^ 2' 



l-2-c(c-f-l) 

+ ... 



(1) 



l-2-3-c(c-hl)(c-f2) 
this series being convergent for \z\ < 1. 

Many physical problems lead to integrals which can be 
expressed in terms of hypergeometric functions, and many 
important functions employed in analysis are merely special 
forms oi F{a,b ;c ;s). Thus : 

(l+^)« = F(-n,y8;^;-^) 
log(l+^) =:zF(l,l;2;-z) 
/„(.)=lim i(^)'L;.^...„u-1._^^ 



n«r(v-|-l) 






+ 1; 



The purpose of this paper is to describe a method for 
computing F(a,b ;c ;^) from ( 1 ) when a, b, c, and z are each 
of the form x-\-iy;x,y real, t^ = — 1. We were confronted 
with these functions through the problem of internal con- 
version of y-rays emitted by a radioactive nucleus. The 
radial integrals arising from the calculation of transition 
probabilities were expressible in terms of hypergeometric 
functions. Our problem involved 90 hypergeometric func- 
tions, and on the basis of a preliminary estimate, 99 terms 
of the series were used. Such complex hypergeometric func- 
tions cannot be conveniently tabulated since they involve 
eight variables, and so a method is required which will 
readily yield F(a,b ;c;z) for special values of a, b, c, and 2^. 



Calculations 

We begin by defining 



U = gn-\r ihn = 



(a-\-n) (b-\-n) 



(2) 



(w+1)(c+m) 
where a = A^-^-iA^ ; b = B^-^-iB^ ; c = C^-^-iC^ ', z = z-^-\- iz. 



Then F(a,b;c;z) = I+/0/X+/0/1/2+/0/1/2/3 + 
The expanded form of /„ can be written as : 
"K . .F. 



(3) 






+ 



m] 



(4) 



where i^i = F,(n) = n^-\-n^(A,+B^-\-C^) 
-f- n[A,(B,+C,) - A,{B,-C,) -f B,C, + B,C,] 
+ [A,{B,C,+B,C,) -\- A,{B,C,~B,C,)] (5) 

= M^ -f a^n^ -f- a^n -\- a^ 

F, = F,{n) = n^{A,-^B-C,) + n{A,{B,+C,) 
+ A,(B,-C,) -\- B,C, - B,C,] 

+ [A,iB,C,+B,C,) -A,(B,C,-B,C,)] (6) 

^3 = Psin) =n^ + nH2C,-i-l) -f n(Q+Cl+2C,) 

+ (Q+CI) (7) 

= n^ + d^n^ -\- d-yU -\- d^ 
Thus, our object is to compute (7), (6), (5), (4) and (3) 
in that order. 

Machine Procedure 

The six numbers representing the real and imaginary 
parts of a, b and c are key punched on cards, and the co- 
efficients a^, a^, . . . , d(, are computed by basic multiplica- 
tions and additions. This computation requires eight sets of 
cards which are later combined on the reproducing punch 
to yield a set of master cards for the coefficients of the 
polynomials in n. The layout for this master set is given 
below : 





Master 


Set 9 


Card Columns 


Data 


Remarks 


1-2 


group number 


each F(a,b ;c ;z) defines a 


3-10 


^2 


group; we required 90 


11-18 


«i 


values of the hypergeo- 


19-26 


flo 


metric function 


27-34 


h 




35-42 


K 




43-50 


K 




51-58 


d. 




59-66 


d, 




67-74 


do 




79 


'X' 




80 


set number 9 
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Computation of Polynomials 

A set of detail cards (set 10) containing n, n^, and n^ for 
n = 0(1)^ is generated on the reproducing punch. This 
set contains ( k-\- 1 ) cards for each hypergeometric function 
to be evaluated, k being the highest value of n used. In our 
calculation each group contained the same number of cards 
(i.e., the same value of k was used throughout) to minimize 
the handling of cards by the operator. 

The master set 9 cards are inserted in front of their 
appropriate groups in detail set 10, and three separate 
group multiplications are performed on the 602-A calcu- 
lating punch to yield F^, F^ and F^. The planning chart and 
control panel wiring for F^ is shown in Figure 1 . 

Sign Control in Group Multiplication 

Since the coefficients on a master set 9 card may be posi- 
tive or negative, their sign must be retained in the machine 
for the complete group of detail set 10 cards following the 
master card. This is achieved by dropping out pilot selectors 
4 to 7, which control the sign of multiplication, through the 
transfer points of pilot selector 1. Pilot selector 1 is picked 
up at the control brushes by the master card and is dropped 
out in the normal manner. 

Computing FJF^ and FJF^ 

Since the polynomials in n can have from 6 to 12 signifi- 
cant digits, 12 columns are assigned to them. Detail set 10 
cards are sorted on F^ into separate groups which have 
12, 11, 10, 9 and 8 significant digits, respectively. Treating 
each of these groups separately, the above quotients are 
easily formed through basic division. The layout of detail 
set 10 is given below : 

De;taii, Set 10 

Card Columns Data Remarks 

1-2 group number n, n^ and n^ reproduced 

3-4 n from card table of n'^ 

5-8 n^ 

9-14 «3 

15-26 F^ 

27-38 F^ 

39-50 F, 

51-58 FJF^ 

59-66 FJF, 



79-80 



set number 10 



Computation of fn 

From equation (4) it is seen that the computation of /„ 
requires a complex multiplication of {FJF^) + t {FJF^) 
by (^1 + i Z2), and this is equivalent to four real multipli- 
cations grouped together as shown in (4). The quantities: 
group number, n, FJF^ and FJF^ are reproduced from 
set 10 into a new set, 11, of detail cards. The values of 
.Si and z^ are key punched into a new set, 12, of master 



cards. By performing a complex group multiplication from 
set 12 to set 11 as shown in Figure 2, the values of /« are 
generated. In our case z^ was positive, and z^ negative for 
all the groups, so that sign control on group multiplication 
as shown in Figure 1 was unnecessary in this operation. 

Consecutive Complex Multiplication 

Having obtained the /„ in the previous operation, we now 
require the products /„, /o/i, /o/i/2> etc. The method used for 
this computation is given below in schematic form : 



■dNo. 


Quantity Read from Can 


i Operation 


1 


/o = go-^iK 


l'fo = Ro+iI. 


2 


/i = gi-\-iK 


fof^ = R^-\-iI^ 


3 


U = 92-\-iK 


fofJ2 = R2-\-ih 



The planning charts and control panel wiring for this oper- 
ation are shown in Figures 3 and 4, pages 165, 166, The 
essential features here are the retention of the calculated 
result from one card to act as a multiplier for the following 
card, and the conversion to true form of this multiplier if it 
should turn out originally as a complement figure in the 
counter. (The machine ordinarily converts complement 
numbers during the punching operation only.) In addition 
to this we must "remember" the sign of, say /o/i> when we 
multiply it by fz to form f of if 2- The scheme is started by 
feeding a blank card under the direction of a separate con- 
trol panel which reads a 1 into storage 4R and resets all 
other storage units, counters and pilot selectors to normal. 

The panel of Figure 4 is then used with one group of set 
11 cards. The first card of this group reads into the machine 
the numbers /o = ^To-j-i/^o and has punched on it /« = Ro-\-iIo- 
The quantities go and ho are retained in the machine and 
are used to multiply f^ = gi-\-ih-^ from the second card 
which has punched on it /o/i = R^-{-iI-^, and so on. 

At the end of program step 6, counters 1, 2 and 3 contain 
Rjc, the real part of (/0/1/2 • • • • fk)- li Rk h negative, the 
counter group will contain the complement of R^- On pro- 
gram 7, pilot selector 3 is picked up by an NB impulse, is 
transferred on program 8 and is dropped out on program 7 
of the following card. Thus, the sign of Rk is remembered 
both during the conversion of Rk on program 8 and the 
multiplication by Rk on programs 2 and 6. A similar proce- 
dure is used for Ik. 

The final step in the calculation oiF{a,b ;c;z) consists in 
summing i?„ and In separately on the accounting machine 
and recording the value : 

F{a,b;c;z) = ( 1 + X)^")"^ *(^^") ' 
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Figure 1. Cubic Polynomiai, (Sign Control in Group Mui^tipucation) 
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Figure 3. Consecutive Complex Multiplication 
Card 1 : /o = 6^0 + ^K = ^o + ^h 

Card 2 : A = ^r, + i/^^ ; /J, = ^i + ^'/^ 
Card 3 : /a = ^Ts + ^^2 ; /0/1/2 = ^2 + ^'^2 
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Figure; 4. Consecutive; Compi^kx Multiplication 



Checking of Computations 

The coefficients a^, a^, . . . , c?o of the polynomials in n are 
checked by manually checking one or two cards, performing 
the machine operation twice, and testing the resulting 
punches for double punching. 

The polynomials F^, F^ and F^ were checked separately 
by summation on the accounting machine according to the 
following formula : 
fc fc fc fc 



=[^^]+4 



+ «o(^+l) 

k(k-{-l){2k-\- 



"] 



+ 



R(^+l) 1 . ,,^1, 



where ^ is the last value of n in the series. F^ and F^ are 
checked in a similar manner. 

Since the third differences of a cubic polynomial are con- 
stant, an alternative check consists of finding 

AF,{n) = F,{n+1) - F,{n) 
A'F,{n) = AF,{n+l) - AF,{n) 
A'F^in) = A'i^i(w+1) — A^F,{n) = constant. 

Generating Differences 

Generally, functions which are tabulated at equal inter- 
vals of the argument can be conveniently checked by taking 
differences up to an order which is sufficient to show a 
"jump" indicating an error. 

For this reason a planning chart and control panel wiring 
scheme is shown in Figure 5 for finding first differences of 
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any tabulated function. Second differences are formed by 
operating on the first differences, and so on for the higher 
orders. The key to the scheme shown in Figure 5 consists 
in "alternating" a pilot selector so that it is transferred for 
the first card cycle, normal for the second card cycle, trans- 
ferred for the third card cycle, and so on. 

The complex group multiplication shown in Figure 2 
was checked by summations on the accounting machine, 
using the normal check for group multiplication. The suc- 
ceeding operations are each performed twice after one or 
two cards are checked manually, and the punched results 
are then tested for double punching. 

The time required to calculate 90 hypergeometric func- 
tions containing 99 terms each is about 140 hours. 
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DISCUSSION 
Mr. Lowe: This problem of the presence of complement 
numbers in storage which are needed in multiplication 



seems to be of quite general interest, and I thought I might 
mention something of which seemingly not too many people 
are aware. 

There is a device available which solves that problem 
fairly neatly. A relay is attached to whatever storage unit is 
specified, which picks up if a complement number is read 
into the storage unit. When the unit is read out, the com- 
plement number is converted so that the true number is 
available either for multiplying or for transferring back to 
the counter. 

Mr. Gellman: I should have mentioned a little more 
clearly, I suppose, but I am sure everyone realizes that the 
602-A converts complements only during punching. It 
punches a true figure, but the complement figure remains in 
the storage. 

Mr. Bisch: I think Mr. ^ellman's example is a very good 
illustration of a point which would interest engineers. He 
has completely systemized the problem to the point where 
the work can be done by someone who doesn't understand 
anything about it. Thus, computers can carry out the ma- 
chine calculation, and the only work they have to do is to 
execute one particular step with care before they go to the 
next. In this way he is able to free himself of the routine 
calculations and can devote his time to the more important 
task of planning new problems. 
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COMPLEX polynomials, or real polynomials with com- 

plex roots, of the form y a^-i x^ = are common com- 

puting problems. In aircraft engineering they arise fre- 
quently in studies of airplane flutter and vibration. 

A successful method of solving these polynomials with 
accounting machines should meet certain requirements : 

1. It must be completely automatic and reasonably fast 
and easy to operate. 

2. It must permit simultaneous attack on all roots. 

3. It must be independent of the order of the equations 
to be solved. 

4. It must be able to accommodate a wide decimal range 
to permit solution of equations whose roots vary in 
order of magnitude. 

The system presented here satisfies requirements 1, 2, 
and 3, and it is satisfactory for many purposes with regard 
to requirement 4. 

Theory 
Newton's method of successive approximations where 

is the basis of this system. /(^») and f (Xn) are calculated 
by synthetic division. 

The synthetic division process may be illustrated using a 
third degree equation : 

f(x) = a^x^ -\- a^x^ -\- a^x 4-03 = . 
Then, writing only the coefficients and letting Xn be some 
complex number for which j{xn) and f{xn) are desired. 



«0 »1 

&o = do, ^1 = «i + &o^« 

Cq =■ Oq, Cj = 0^ -\- C^Xn 



^2 = «2 + ^i^n, 
f(Xn) = b2-{- C^Xn 

tts 
f(Xn) = ^3 4- b^Xn . 

(2) 



*This paper was presented by title. 



As discussed later, f(xn) and f'{xn) are calculated in one 
run through the IBM Type 602- A Calculating Punch, and 
Xn+i in a second run. 

Using a standard, full-capacity 602- A, it seems that the 
real and imaginary parts of Xn can be expressed to six 
digits each and the complex quantities a^, bi and c^ to eleven 
digits (twelve if the storage unit conversion device is avail- 
able). 

Since all numbers must be handled according to some 
fixed decimal point, it is frequently desirable to effect a co- 
ordinate transformation, x' = ex, such that the largest root 
of the transformed equation is near unity. Such a trans- 
formation permits groups of dissimilar equations to be han- 
dled simultaneously with a fixed decimal point. If, as is the 
case in airplane flutter analysis, the equations are charac- 
teristic equations of complex matrices, it is advantageous to 
make this transformation on the matrix. Incidentally, such 
a transformation on the matrix greatly facilitates the proc- 
ess of deriving the characteristic equation. 

An automatic application of Newton's method, as de- 
scribed here, will not always find all roots from first 
approximations. The difficulties are discussed in many 
numerical analysis texts. On the other hand, it is always 
possible to obtain some of the roots and, thus, by manual 
analysis, arrive at better approximations to the missing 
roots. These can be used as the basis for further machine 
iteration. 

Another approach to the problem of missing roots is to 
factor known roots from the original equation (this can be 
done by the synthetic division process), and operate on the 
fc— 1 

reduced equation V^ bk-i x^ — 0. In fact, if nothing is 

i=0 

known about the roots, it may be advisable to start with an 
initial approximation of x^ = 0, find a root, factor it out 
and repeat on the reduced equation. However, caution 
should be exercised in factoring out large roots, as it may 
be necessary to know such roots to a relatively large num- 
ber of significant figures to obtain an accurate reduced 
equation. 
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Proc^Sdur^ 

One detail card is punched for each complex coefficient, 
Oi, of the equation. These are arranged in order of subscript, 
constant term last.j A master card containing Xn is filed in 
front and a blank trailer behind. The group is run through 
the 602-A, the master and detail cards skipped out without 
punching, and complex j{xn) and f{xn) punched in the 
trailer. The detail cards are sorted out, and the master and 
trailer run through the 602-A again to calculate jr„+i by 
equation ( 1 ) . Also, in the second run, an 11 is punched in 
the trailer to make it a master. It may then be filed in front 
of the detail, a new trailer placed behind, and the cycle 
repeated. 

If a number of equations are being handled simultane- 
ously, the cards may be collated after Xn+^ is calculated and 
both cards pulled if Xn = ;f»+i. By match-merging the detail 
behind the remaining new masters, roots which have con- 
verged will automatically be eliminated. 

Obviously, any number of groups for one equation can be 
going through the pi-ocedure simultaneously. Thus, one or 
more approximations to each root can be handled at the 
same time. 

A good check on the work is provided by listing both 
masters after .*•„+] is formed. If f(x) is decreasing and x is 
not changing rapidly, it can be assumed that the process 
is working correctly. 

Thus, the operations are seen to be extremely simple and 
can, in fact, be handled by an operator with minimal knowl- 
edge of machine accounting. This leaves a discussion of the 
two 602-A control panels. 

The synthetic division control panel* can be explained by 
reference to (2) 

Master Read-in new x and clear old f(x) and f{x) 

flo detail xO -\- a^ = bo jt'O + = 

Oi detail xb^ -\- a^ = b^ xO -\- b^ = c^ 

a^ detail xb^ -\- a^ = b^ xCq -j- 6^ = c^ 

ttg detail xb^ -\- a^ = f{^) xc^ -\-b^ = f(x) 

Trailer Punch /(;ir) and f '(.«■) 

Note that the formation of both bi and Ct follows the same 
pattern. If bi is calculated and the program unit allowed to 

»The writer will supply copies of the layouts for the 602-A control 
panels on request. Address John Lowe, Douglas Aircraft Com- 
pany, Inc., Engineering Department, Santa Monica, California. 



repeat itself, d can be formed with little selection of pro- 
gram exits, 

flo is often equal to unity. If this is true for a particular 
group to be handled, b^ = I can be emitted into storage on 
the master card, and the a^ detail card omitted. 

The quantities bi and Ci, which are often negative, must 
be placed in storage units and used as multiplicands. Hence, 
the wiring can be simplified, time saved, and one more digit 
carried if the 602-A is equipped with storage unit conver- 
sion devices. 

The control panel to form Xn+i by equation ( 1 ) is straight- 
forward. The square of the modulus of f{x) is formed, 
f{x) is multiplied by the conjugate of f{x) and the two 
indicated divisions and subtractions performed. Xn-^ can be 
calculated to eight decimal digits, although only six can be 
used in the synthetic division. This may be desirable as 
when the root has converged to six digits, the next approxi- 
mation may be accurate to seven or eight digits. 

If both f(x) and f{x) are small, the products of these 
by the conjugate of /'(.*:) may be very small, and the quo- 
tients may have insufficient significant figures. In this case, 
the trailers may be reproduced before the calculation of Xn+i 
multiplying both f{x) and /'(•*■) by a power of 10. Obvi- 
ously, the same device will work if f(x) and f'{x) are too 
large. 

Factoring out a root and punching the reduced equation 
may be accomplished simply. The coefficient cards are re- 
produced, omitting amounts and common punching. These 
new cards are filed behind the original coefficient cards by i. 
The master containing the root is filed in front, and the 
entire deck is run through the 602-A. The blanks serve as 
trailers and receive the coefficients of the reduced equation, 
the amount in the last one being the residual and providing 
a check on the work. The 602-A control panel for this 
operation accommodates eight digits of x, uses the same 
synthetic division process described above, and is quite easy 
to wire. 

As an example of time required to obtain roots by this 
method, consider a group of sixth degree complex equations 
with coefficients given to not more than eleven (or twelve) 
decimal digits. Evaluation of f{Xn) and f{Xn) for one root 
requires about one minute. Thus, if the roots converge after 
an averageof five iterations, it requires about one-half hour 
per equation to obtain all the roots. 
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THE PURPOSE of this paper is to give an account of 
experience in the inverting of matrices of high order, 
obtained in connection with the engineering problems of 
North American Aviation. Because information on this 
subject for really large matrices (i.e., for « >> 10) is 
notoriously sparse in the literature of the subject, and be- 
cause of the great importance of the problem in many fields, 
such a paper should be of interest to this seminar. In addi- 
tion, it is hoped that research organizations will be stimu- 
lated to make investigations, which we, not being a research 
organization, are neither qualified for, nor encouraged to 
carry out. 

When simultaneous linear algebraic equations are met in 
engineering work, as they frequently are, it is usually nec- 
essary to solve a number of systems having the same co- 
efficients, but different constants on the right-hand sides. 
It is shown in reference 1 that if the number of systems is 
at least four (i.e., if there are at least four columns of con- 
stants), it is economical to compute the inverse matrix. In 
our work this is generally the case, and for this reason ma- 
trix inversion is important. 

It is not practical to perform the inversion process manu- 
ally (with desk machines) unless n < 10, and even for 
these small matrices, if it is necessary to invert a large 
number of them at any one time, the work is done much 
more quickly, and accurately with the help of IBM. There- 
fore, both systems of high order (w > 10) and large groups 
of small systems are handled by the IBM group. 

The small systems are handled quite successfully by a 
variant of the well-known Aitken method. To give an idea 
of the efficiency with which this work is done, 48 fifth-order 
matrices were recently inverted and checked in 16 hours — 
an average of 20 minutes per inversion. This work was 
done using the IBM Type 602 Calculating Punch; use of 
the IBM Type 604 Electronic Calculating Punch would 
cut down the time somewhat, but not greatly so because of 
the large amount of card handling involved. Although this 
is far from being the most efficient use of the machines, 
those with experience in numerical inversion will recognize 
it as being amply justified. 

The large systems present special difficulties which re- 
main to be solved. Not only does the number of operations 



increase enormously with the order, making the process 
very slow, but also such systems rapidly tend toward in- 
stability as the order increases — i.e., the rounding errors, 
which are inevitable — accumulate in a serious way. In this 
connection, see references 1 and 2. 

Because of the inherent instability of the direct methods, 
several well-known iterative methods (classical iteration, 
and the method of steepest descents) were tried. For large 
systems convergence of these methods is much too slow. 
Convergence is theoretically assured for positive definite 
matrices. Positive definiteness was guaranteed by the expe- 
dient of taking A' A, where A' is the transpose of A, the 
matrix in question, and 

A-^ = {A'A)-K4'. 
Despite the theoretical assurance of convergence, in this 
case, numerous iterations showed no evidence of conver- 
gence at all ! We probably did not give sufficient trial to this 
approach, but our negative results with the methods tried 
are confirmed by other investigators.^ Iteration has its value 
in the improvement of an approximate solution obtained by 
some other means. 

In choosing among the many direct methods, several con- 
siderations are important : ( 1 ) The number of elementary 
operations should be a minimum. (2) The arrangement of 
work and order of operations should be convenient with 
respect to the peculiarities of the machines. 

Concerning the first requirement, it is asserted in refer- 
ence 2 that the elimination method requires fewer opera- 
tions than other known methods. There are numerous 
methods, however, which can be considered to be but slight 
variations of the elimination method. The method of par- 
titioning of matrices, for example, is a generalization of the 
elimination method, and the various methods which involve 
pivotal reduction — those of Aitken, Crout, Doolittle, Gauss, 
etc. — are closely similar and require about the same number 
of operations. The method of determinants is an example 
of a method definitely inferior to those mentioned above. 

With respect to the second requirement, suitability for 
the machines, methods which include such things as re- 
peated cross multiplication are to be avoided. 

*This paper was presented by title by Paul E. Bisch. 
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A direct method which fits these requirements is a new 
variant (as yet unpubHshed) of the ehmination method de- 
veloped by Mr. Charles F. Davis of our IBM group. Al- 
though this method has several features to commend it for 
use with IBM machines (and with desk machines as well) 
it is not claimed that the successful inversion of several 
large matrices could not have been achieved by other meth- 
ods. The method used was simply the elimination method 
with some new twists. Since the details are still being 
written up they cannot be given here. 

The point of most interest is that inversion of a matrix 
as large as 88 by 88 has actually been carried out satisfac- 
torily, using standard IBM equipment. The prevailing 
opinion among the authorities is that the inversion of a 
matrix of this order is a practical impossibility (cf. refer- 
ence 3, page 2 and pages 6-8). In a sense this may be true, 
for this first attempt at inverting an 88th order matrix took 
about nine weeks and involved between 60 and 70 thousand 
cards. The better performance which should come with ex- 
perience might still be prohibitively long and expensive 
from the engineering point of view. Nevertheless, the de- 
gree of success we have had seems hopeful. 

A word of explanation should be given about what we 
have considered "satisfactory" in the way of accuracy. Un- 
fortunately this is a difficult question if one demands 
precise limits. The question might be phrased this way ; If 
a solution of a linear system is substituted in the original 
equations and all the remainders are small, is the error in 
the solution small ? How small ? If the system is 

AX —B =^0 
and a solution X^ is substituted there results a column of 
remainders R^ 

AX, — B^R, . 
Elementary matrix theory gives the answers to the above 
questions in terms of the norms of the column vectors R, 
and B, — X — X,, and the quantities A and ju,, the upper 
and lower bounds, respectively, of the matrix A. 

l/X\R,\^\B,\<l/t.\R,\ . 
From this we see that if A is large and /a small, the limits of 
error can be very wide ; in particular if ju, is sufficiently 
small, |Hi| may be large, although \R-,\ is small. Thus, 
what is often considered to be good check may conceal large 
errors. The main difficulty, however, is that the quantities 
A and ju, are not known, and the work required to find good 
estimates of them is usually prohibitive. 

We have been obliged to get around this difficulty in a 
manner rather unsatisfying to the mathematician but wholly 
acceptable to the engineer. The engineer looks at the nu- 
merical results, and, with physical intuition as a guide, de- 
cides whether they are reasonable. To take an example, the 
solution of a set of 66 equations gave us the stress distribu- 
tion of a sheet stringer combination. The regularity of the 
results and confirmation of what should be expected on the 



basis of experimental evidence convinced engineers that 
this was the "right" answer. -Whether a given numerical 
value found in this way is correct to two, three, or more 
significant figures is not known. 

There is still another rough indication of accuracy. The 
experienced IBM operator working in a fixed field (8 digits 
in our case) can fairly well tell when things are behaving 
nicely or not. Common to all variants of the elimination 
method is a division at each reduction. The continued prod- 
uct of these divisors is the determinant, and although the 
determinant is large, some of these divisors may be small. 
The occurrence of small divisors means the loss of signifi- 
cant figures, i.e., the process blows up. In the reduction 
process of the 88 equations, as well as the 66, this difficulty 
was not apparent. Furthermore, iteration of solutions ob- 
tained showed convincing evidence of rapid convergence. 

Strictly speaking, the inverses of these large matrices 
were only partially determined. The earlier statement that 
it is economical to compute the inverse when there are more 
than four columns of constants needs qualification. It is not 
strictly true for the matrices of quasi-diagonal character 
(explained later) dealt with by us. However, the process 
effectively leads to a decomposition into diagonal and semi- 
diagonal factors, from which with some additional work the 
inverse can be found explicitly. 

It is true that no general conclusions can be drawn from 
such limited experience as ours with matrices of a particu- 
lar type. But matrices of this type occur frequently in struc- 
tural analysis and elsewhere. The matrices spoken of may 
be considered to be made up of the finite difference approxi- 
mants of linear partial differential equations. Each stepwise 
approximant has the important feature that coefficients of 
all but a few of the unknowns are zero. Thus, these coeffi- 
cients can be arranged in such a way that the matrix has 
large triangles of zeros in the upper right and lower left 
corners. Such a matrix might be called quasi- diagonal and 
is certainly one of the most important types. Since the limit 
case, a diagonal, is stable (unless some diagonal element is 
zero) and trivially easy to invert, it is plausible to suppose 
that quasi-diagonal matrices are particularly stable. It is 
suggested that our experience lends weight to this supposi- 
tion and that a study of quasi-diagonal matrices will yield 
much more optimistic estimates of precision than are found 
in the literature. 

On page 1023 of reference 2, von Neumann and Gold- 
stine conclude that, "for example, matrices of the orders 15, 
50, 150 can usually be inverted with a (relative) precision 
of 8, 10, 12 decimal digits less, respectively, than the num- 
ber of digits carried throughout." By "usually" is meant 
that "if a plausible statistic of matrices is assumed, then 
these estimates hold with the exception of a low probability 
minority." These conclusions, which are the result of a 
thorough analysis, are valuable to those who are anticipat- 
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ing the automatic computing machines of the future, but to 
those who think it might be practical to invert certain types 
of large matrices, using standard IBM equipment here and 
now, they seem unduly pessimistic. 

The critical question is that of "a plausible statistic of 
matrices." The estimates of von Neumann and Goldstine 
are made in terms of A and /x, the upper and lower bounds, 
respectively, of the matrix — quantities not known in ad- 
vance and very difficult to determine. The numerical esti- 
mates quoted above are the results of introducing statistical 
hypotheses concerning the sizes of these quantities. This is 
done in the form of the results of V. Bargmann concerning 
the statistical distribution of proper values of a "random" 
matrix. It is possible that a similar study of the quasi- 
diagonal type, defined in this paper, might lead to less dis- 
couraging conclusions. It is stated in reference 1, page 59, 
that these estimates "can also be used in the case when the 
matrix is not given at random but arises from the attempt 
to solve approximately an elliptic partial differential equa- 
tion by replacing it by n linear equations." But no reason is 



given, and perhaps this manner of stating the point indi- 
cates a certain lack of conviction. However, the original 
source of these estimates is not accessible to this writer. 

To summarise : On the basis of limited experience invert- 
ing matrices, it appears to us at North American that, con- 
trary to prevailing opinion, it might be practical to invert 
certain important types of matrices of high order, using 
standard IBM ^uipment. What is needed is further statis- 
tical study of these\types, and, if the estimates of precision 
so obtained are favorable, a comparative study of known 
methods of inversion. 
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